Mike Hudy – Using technology to integrate employee selection and candidate recruitment

Steve

Hi, this is Steve Hunt with People Performance Radio. This week we spoke with Mike Hudy from Shaker Consulting about a tool they have called the Virtual Job Tryout, which is a really advanced tool in terms of selection of employees, whether for external hires or internal promotions, and it’s a tool that takes beyond the idea of giving people tests and the traditional methods to more really giving people a chance to try out a job before they go into it, “a very highly interactive experience” is the term that Mike uses, that people go through, and you collect information from them, but you also give them a real sense of, this is what the job involves, these are the sort of tasks you’re going to do, these are a variety of different types of selection tools, and Mike talks about the Virtual Job Tryout, talks about how it’s different, and how companies are realizing some really astronomical business value from it, so it’s an interesting conversation about a very large step forward in the field of selection science, and something that I think we’ll be seeing more and more of over the coming years, so let’s listen to Mike Hudy from Shaker Consulting.

Hi, this is Steve Hunt, you’re here with People Performance Radio, and today we’re talking to Dr Mike Hudy, who is with Shaker Consulting. He’s going to talk to us a little bit about what they call the Virtual Job Tryout, so Mike, thanks for joining People Performance Radio.

Mike

Thanks Steve, appreciate it.

Steve

So, what is Shaker Consulting, what’s your background, just before we jump in?

Mike

Shaker Consulting Group is a firm that really specializes in measuring the people side of the business, and more and more we’ve gravitated to this product of ours called Virtual Job Tryout, which we’ll get into more later, but it’s really a very different way of looking at pre-employment assessment, and the vast majority of our business now revolves around developing, validating, deploying these Virtual Job Tryout systems to make better decisions about talent. The majority of those decisions are around selection, sometimes the system’s used for promoting and career pathing, and even more, getting more into developmental feedback now.

So that’s a little bit about Shaker, my background is PhD in industrial organizational psychology, I’m one of the partners here at Shaker Consulting Group, Shaker was formed about seven years ago, and that’s a little bit about me.

Steve

Cool. So the Virtual Job Tryout, as best as you can describe it over the radio here, what exactly is it? – so it’s a tool, it sounds like it’s used for, you give it to people being considered for certain jobs, either external candidates or internal candidates, but what are they actually going through? – is it a test?

Mike

Yes, that’s a good question, I like the way you presented it as well, as best you can over the radio here, and that is the thing about the Virtual Job Tryout, is it is a very visually stimulating, engaging experience that is difficult to put into words, and it’s kind of one of those things that you need to see a demo to actually say, “aha”, I get what it is, but as best I can in words, is it a test? – we shy away from ever calling it a test, we don’t call it an assessment either, we call it an experience, because it is designed with the candidate in mind, most of the assessment space, when they develop tests or assessments, it’s all about the data I’m going to collect, the construct I’m trying to measure, and what gets left behind is, well what about the candidate?

We take a different approach to it and say that actually we want to collect information about the person, but what a great opportunity to begin building the relationship with that individual, and to convey as much information as you’re getting from the person, and so a lot of care goes into developing the Virtual Job Tryouts to provide information about the organization, about the job, some of the key challenges, the things that people don’t necessarily like about the role, so that a candidate at the end of the experience actually gets something out of it, they walk away rather than just having a sore wrist from clicking in a bunch of bubbles, they actually walk away with a better understanding of that organization’s culture, who they are, and the position, so they can determine, is this right for me? Do I want to continue down this path of exploring an opportunity with this organization?

Steve

Can you share any specific clients and jobs that have used it, and I have a follow on question?

Mike

That’s a good question, in that we’ve done Virtual Job Tryouts across a lot of different industries and a wide variety of jobs, so you immediately think of jobs that are entry level, high volume, which is certainly a sweet spot, and we’ve done plenty of Virtual Job Tryouts there, but when we’re doing jobs like field service technicians and district managers and insurance agents and mid-level professionals, what we’re looking for is jobs where hiring is strategically important, either you’re doing a lot of them, or when you make mistakes it costs you dearly, so we have one client who built a Virtual Job Tryout for a capital equipment sales position, they only hired ten a year, but they’d made the investments through doing a business case analysis to say, every time we get it wrong, it costs us millions of dollars, so if by using a Virtual Job Tryout we’re able to make one better hire over the course of the entire year, we’ve paid for the system 30 times over.

So a variety of positions, a variety of industries, and I think the other common theme when we look across who we’re working with, it’s companies where people really make the difference, the companies that are, people are strategically important to delivering the value, so companies like Starbucks, that you’re going in and you’re paying a bit of a premium for their product, but part of what you’re getting is the Starbucks experience when you go into the stores, the Starbucks experience is largely created by the people that work in those stores.

Steve

Let’s maybe use that, maybe Starbucks is a good example, let’s say I’m applying for a job, and I go through the Virtual Job Tryout – what am I actually going to see as a candidate?

Mike

You go to Starbucks’ career page, and they have a talent management system like most big organizations do, and you begin the process there by creating a profile, and one interesting note there is, before implementing the Virtual Job Tryouts, Starbucks used to ask 30 plus questions in their talent management system, and through the development process of the Virtual Job Tryout, we included the most commonly asked questions in our validation process, and found out that most of them weren’t actually linked to performance and many of those screening questions actually had a reverse relationship, and so we helped them streamline, these are the seven questions that are linked to on the job success, so they asked them sort of background and experience questions, and Starbucks’ goal was, to because the Virtual Job Tryout is such an engaging experience, we want to get them as quickly as possible into the Virtual Job Tryout, so they spend very little time in the talent management system, and then they get to the last page there, and it’s a description of the next step is the Virtual Job Tryout, click here to begin your experience, and there’s data integration built between their talent management system and our Virtual Job Tryout, so candidate data is transferred over to our system, there’s no log in, and they go right into the Virtual Job Tryout, go through the Virtual Job Tryout experience.

Steve

Mike, just one thing, really specifics I know, our listeners are wondering, I know I’m wondering, can you just, when they go into the Virtual Job Tryout, what do I see? Is it a video, are they questions? – what would I actually see, what would be on the screen?

Mike

Yes, the Virtual Job Tryout is a multimedia experience, we use audio, we use imagery that kind of fades in and fades out of the screen, there’s a narrator who kind of guides you through the whole experience, and it’s set up as, “You’re going to have a chance to take the job for a test spin, during this Virtual Job Tryout we’re going to let you take the job for a test drive, and so it’s not set up as we’re going to assess you, we’re going to test you to think, to see if you’re good enough, but we here at” (to use Starbucks again as the example), ”we here at Starbucks want to tell you as much as we can about the organization, and about the store manager job, so what better way to learn about the job than to try it out for 30 minutes or 45 minutes, and do some of the things that our store managers do day in and day out.”

Steve

So what would be an example of something that sort of person would do, do they say, “Here’s an employee that’s got a problem, you have to give him some feedback?”

Mike

Yes, it varies from system to system, because each of our Virtual Job Tryouts are tailored or custom-built to the job, and so it begins with the process of, what are those important, those key elements of the job that we need to represent in the Virtual Job Tryout, so certainly as a store manager one of the key things that you need to do is you have a staff to manage, you have customers, and so those interpersonal interactions are a huge aspect of the role, and so there is an experience that puts candidates into store manager situations where they hear interaction between store managers and customers or store managers and direct reports, and they’re asked how they would respond in those situations.

Another example, using the store manager, it’s their business, they’re running in some cases a multi-million dollar business, and so they rely on reporting and metrics to really drive, “Where’s my business off, what changes do I need to make”, and so we’ve simulated that as well, that, “OK, now you’re a store manager and you’re running store number 1579, and your store has some strengths, it has some limitations, and here’s some data around your store – what are you going to do? What are you going to prioritize? What are you going to work on to increase the performance of a store?”

Steve

So it sounds like it’s, what used to be called an assessment center, but it’s a highly automated version of an assessment center where people are doing a variety of different exercises that are giving them both a sense of what it’s like to be in the job, but are also collecting information around their ability to actually do the job.

Mike

I think you’re dead on with that analogy, and we’ve used it ourselves as well, it’s similar to an assessment center, and you’re relying on multiple different measures, and that’s really one of the, in addition to the benefits to the candidate, the benefits from the measurement side of things is, it is a multi-method measurement, meaning that within any one Virtual Job Tryout, we have five, six, seven different types of psychometric measures that we’re using that are all giving different kinds of data that are then being combined, and that’s one of the reasons that assessment centers are so successful and so predictive of performance, because they’re relying on multiple different methods to collect that data, and we’ve built that concept into the Virtual Job Tryout.

Steve

So that’s cool, so you’ve integrated multiple measures together, so the candidate’s going through and getting this rich experience of, OK, this is stuff you need to do in the job, at the same time the way they respond to that experience you’re tracking from different perspectives in terms of their financial skills, their management skills, interpersonal skills.

Now, when you look at the end of the Virtual Job Tryout, when you think about, what do we judge a success? You mentioned, well, one, we want it to be predictively valid, we want it to predict job performance, we also want to give a positive candidate experience. What are some of the things that a company gets out of using a Virtual Job Tryout that goes beyond what they’d get from more typical personality or ability selection tests?

Mike

A couple of things there, first I’ve already alluded to, we ask for feedback at the end of the experience, and that’s pretty unusual, because most wouldn’t want to hear the response from a candidate who just filled out 30 pages of personality questions, “What did you think of the experience?”, so that’s something unique, and we report that back to our clients, and there is research out there that says that candidate reactions are statistically tied to some important outcomes, like the likelihood of actually accepting the job, or the image of the company that you have after the experience, or, if you don’t accept the job, your likelihood to come back again in six months or a year when you’re thinking about changing jobs.

So that’s one thing that’s a little different, but then where we’ve also tried to innovate is, an assessment doesn’t have to be just an assessment, it can be this full experience from the candidate’s side, so innovation from the candidate experience, but also from the decision support side as well, and so what we’ve done is, we have some typical competency fit reports, we think that ours are cooler and neater than others, but most assessment systems are going to give you something like that, but where we’ve gone is really tried to be end-user centric from who’s using the report, so we’ve talked to recruiters, we’ve gotten feedback, “What would you like beyond this competency profile?”, and the reports that we’ve developed aren’t necessarily the ones that, the scientists at Shaker Consulting would have thought of, but the people who are out in the field, and are actually doing the job, and are processing hundreds or thousands of candidates, came back with feedback that, “We want decision support tools that are going to help us further explore the candidates”, so we’ve come up with a variety of different dynamically-generated reports that recruiters can, “OK, I like what I see here, I’m going to schedule a phone interview, and now, instead of just using my standard set of questions, I already know the areas that I’m going to probe into – this is a strength, this is a weakness, here some red flags that popped during the Virtual Job Tryout”, so those are some of the reports that we’ve developed.

A newer one that we’ve developed is what we call an Individualized Marketing Plan, so yes, recruiters are evaluating talent, but they also have this role of selling the organization and selling the position to candidates, and assessments systems don’t give recruiters anything there, so we’re learning a lot about candidates that can be used to tailor the way you communicate about the job, and actually motivate them and get them excited, so I find out that they’re interested in the job because of the unlimited earning potential – that’s pretty important for me to play up in an email follow up that I send, or when I’m talking to them on the phone prior to conducting a phone screen, so we’ve come up with this concept of an Individualized Marketing Plan as well, which we haven’t heard about anybody else doing this, and we think it’s something that recruiters are really going to like having and find value in.

Steve

Yes, I think it’s really cool, you look at the whole process of selection, people might say, “Look, the same time you’re evaluating the candidate, the candidate is evaluating you”, so I assume good candidates always have multiple offers, so you’ve really said, “How can we blend these two things, the selection side, the evaluation of the candidate, but also the recruiting marketing side, and put those together into one form?”, which I think is a really innovative approach.

I guess the last question that I have is, what’s the business impact? I know that you guys do tons of validation studies and are going to make sure these things do empirically predict job performance, but you’ve had some up and in place for a while now – what have you been able to show to the line of business non-HR leaders why this system is worth it?

Mike

That’s the really exciting thing in the end, that’s what we’re after, it’s an engaging experience, it looks cool, and we have all the science behind it, but in the end the people that are paying for it, that’s exactly what they’re looking for, and we’ve been able to achieve results that really you can’t with other types of system, and because (a) you have engaged candidates, but (b), what we already hit on is this multi-method measurement, we’re able to statistically model against metrics and business metrics in a way that you can’t with just a personality inventory or just a cognitive ability test, or through simulating some of the things that people need to do to achieve those metrics, we’ve had much more success, stronger relationships with those metrics, and stronger relationships with those metrics of course mean that the return on the back end is going to be greater, so it really depends what job we’re talking about, for some, call center, teller type jobs, turnover being the key criteria, we’ve seen pre and post implementation where turnover has dropped 50%, 70%, so those types of gains.

For other higher level jobs where it’s not about turnover, but it’s more about performance variation and the quality of hire, and if I do a better job of screening out people who are going to do just enough to not lose their job but leaves so much revenue on the table, we’ve seen multi-million dollar differences person by person in, for instance, capital sales, capital equipment sales. One of our clients realized a $24 million year over year gain on just one metric for one job, we presented that, we co-presented that result with the head of staffing for that organization, the business, the head of the business stood up and said, “Finally you got it right, it’s not just about turnover and controlling costs, HR has finally helped me contribute to the bottom line”, so those are the types of things that we’re finding in terms of business impact.

Steve

That’s cool, so basically you’re really demonstrating through this system that if you really do focus on getting the best possible people, which isn’t just evaluating them, but it’s also really fitting them to the right job, getting people in jobs that they can succeed, and in following up you are seeing in fact that people really do make a big difference, if you get the right people in the right place.

Mike

Yes, they make a huge difference, and all too often the return on investment models are built on turnover, and what we’ve seen is that the bigger impact is actually on performance variation, when you said, “hiring the best”, and that’s one way of looking at it; the other way of looking at it is avoiding hiring the worst, so just help me make fewer mistakes, and the salesperson who’s selling $1 million versus $8 million, well I’d like to hire all $8 millions, but if I can’t do that, help me avoid and run screaming from the person who’s only going to sell $1 million and leave $7 million on the table.

Steve

That’s a great point, so it’s both the positive sides and avoiding the negative. Somebody said, “Employees are your greatest asset, but bad employees can be your biggest liability.”

All right, well Mike, thank you so much for talking with us about the Virtual Job Tryout, we’ve been talking with Mike Hudy from Shaker Consulting about what is a very innovative approach to assessment, and I think some of you people who have listened to this podcast know it’s an area that I’ve done a lot of work in myself, and definitely a very cool approach, it’s really pushing the envelope on what selection processes should be about, so Mike, thank you very much for appearing on People Performance Radio.

Mike

Right, thanks Steve, appreciate it.

Speaker

If you would like to be a guest on the show, or sponsor, please drop us a line at podcast@successfactors.com, or you can leave us a message at 650-425-7474. This podcast is copywritten by SuccessFactors. The views expressed are the individual’s own, and do not necessarily represent those of SuccessFactors, SuccessFactors’ partners or customers. See you next week.