Charles Handler - Using Assessments to Improve Hiring Decisions

Steve

Hi, this is Steve Hunt with People Performance Radio. This week's interview is with Charles Handler, who's the president of Rocket-Hire, which is a company that focuses on helping organizations use online assessments to make better hiring decisions, and Charles talks about different kinds of assessments, what you should look at when you're considering using these powerful, but there are some risks to them, tools such as personality tests, ability tests, simulations, to guide hiring decisions. Charles also makes reference to some books and things that you can find to help you learn more about assessments, now if you'd like to find some of those books, they're available on the SuccessFactors' bookstore, which is on our website, and so that's the place where you get additional information as well as we have links, because as you'll hear in this, assesments, while powerful, there's also a lot of information you have to use to make sure you're using them correctly, so for a good overview of assessments, give this a lesson— here it is, Charles Handler.

Hi, this is People Performance Radio, this is Steve Hunt. This week we're talking with Charles Handler, who is the president of Rocket-Hire, a company that focuses on online screening and assessment tools and helping organizations use automated technology and assessment science to make better hiring decisions. Charles, welcome to People Performance Radio.

Charles

Thanks Steve, I appreciate you having me on today.

Steve

So, why don't you tell people a little more about what assessments are? I'm guessing most people aren't even sure what we mean when we say, 'online assessments'?

Charles

Sure, it's a very broad term, and there's a lot of things that could fall into that particular bucket or under that particular name, but in general I look at it as, any indicator that you use during the hiring process to predict how well someone will perform at a job or will fit with an organization, so that's a pretty broad thing, and there's definitely variation in that there are some things that are much more scientific, and that's really where I concentrate most of my work, and that is, scales or inventories of questions that have been shown to measure certain traits or things about an individual, and that can span across personality, that can be their mental ability, cognitive ability, how they think, how they solve problems. It can also range into things such as situation judgement and even life history things, such as things that go on the supposition that someone who, what someone has done in the past is a really good predictor of how they will perform in the future, so it's a wide range of stuff.

Steve

So it's really the type of things like personality tests, intelligence tests, screening questionnaires that ask about what you've done in the past?

Charles

Sure.

Steve

So there are a lot of those out there, what do you do when you're working with companies, if somebody is saying, well, personality testing, there's an article in the Wall Street Journal this week which was talking about issues with that— how effective is personality testing— all these tests, do they work? How can you tell?

Charles

Well I think they do work, I think that we have, as psychologists and scientists, a very good track record of history that show they work, but they don't automatically work if you just decide, oh I'm going to use this one arbitrarily, and plug it into a situation— you really need to do some homework to make sure that the test or assessment that you're using is actually going to be related to the things that you're trying to measure, and I think that's one of the biggest points where people stray in this stuff, and if you don't get that part right, honestly you really, it's everything downstream of that has less of an opportunity to really work well for you, so there is some certain upfront work and some certain understanding that needs to go into it. As psychologists and researchers, we have looked at the relationship between how someone would score on one of these tests and important business outcomes quite a bit, and I would say that we've greatly accelerated our understanding of how these things work, and why they work, and even been able to dial them in to be even more effective, since the internet has become a delivery vehicle for assessments, and that's because we've just been able to get so many data points now that we've really been able to see some clear trends, and it's helping us build a strong case for how well these things can help.

Steve

So what do you see happening right now with these as far as hiring practices and the use of assessments— are more and more companies using assessments, and if I'm a company that's saying, gee, these sound interesting, I want to improve my hiring process— where should they start? What's the first step to say, I want to implement some tests to make better hiring decisions?

Charles

Right, that's a great question. What I do in my business is I use a methodology that really has a couple of steps to it, and the critical step is to really talk to them and do what I call a 'discovery phase' where I really look at what are they trying to accomplish, what jobs are they trying to hire for, what does their hiring process look like, what are the business outcomes that they're hoping to realize through the use of these tools? And it's almost always a more tactical situation that I deal with where people are having, OK we have high turnover in this one job, or we need salespeople who are going to be able to have these attributes and sell in this kind of manner to be able to move this particular product, or something like that, so that's usually the driver, and we look at what's going on with that job, and then also with the organization, what's the culture like there, what's the context these people are going to have to be working in, and define a real solid set of parameters that we can use, and then go out and actually look at what's available, and try to match the characteristics of a specific test, and it often even moves into more of, not just a test, but a software system or some kind of a platform that allows you to manage the assessment process, and even other parts of the hiring process.

That's another thing that I really encourage my clients to think about, and anyone who asks me about this to think about, is the fact that hiring really isn't just one test, so a test or a couple of tests are definitely something that can help you, but if you use those in the middle of a really broken process, they're surrounded by a broken process, or an inefficient process, you're not really going to be able to realize the full maximum potential, so we really look at the entire process and look at everything from how you're sourcing and attracting candidates, what messages are you sending, all the way through the steps that would make the hiring decision, and usually there's a good spot in a couple of places within that flow to be able to say, OK, here's the best type of assessment that would work in this situation.

That's just to help a company actually decide, OK, we don't know anything about this, we want to try and use a particular vendor; the actual implementation of the assessment is a whole another step whereby you do what I alluded to earlier, and really understand the job in a more formal manner, in a process known as 'job analysis' where we really break the job down into the human traits and characteristics, behaviors, what have you, that are required for the job, and we work backwards from that to set up the selection process or the different tests in interviews to make sure we're measuring all of those things as completely as we can.

Steve

So what I'm hearing is a couple of things: one, that when you're looking to say, we want to improve the hiring process, it's a lot more than just going out and buying the test, first you need to look at the whole process, because one thing, if you don't have good candidates, the test is only going to select from the candidates that are available, so you have to make sure you're bringing in good candidates and have it within the process. The second thing is, you really have to understand the job, so this isn't about just going out onto the internet and buying the test, you really need to think through how you're going to use the test and how it's going to fit.

What are some of the main things people need to really focus on as far as, as they're looking at these tests, should they use a personality test, should they use an intelligence test, are there any risks to using these things?

Charles

Oh absolutely there are risks, and I think again, and it's a great concern, I do some survey work every year and that always bubbles to the top, that people are worried about the legal ramifications of this, and even now you're seeing more stuff in the news, as I collect bits and pieces from the web about testing, there's more stuff bubbling up, and you've even got a situation whereby I honestly feel like the changes that are going to go on politically are going to bring a little bit more scrutiny to how people are using stuff. So there's definitely concern, but I feel like, again I'm going to keep hammering away at this one point, that you really look at the job first and understand the job, and make sure that you're aligning everything you're asking in your hiring process to things that are relevant to job performance, and you'll generally be OK.

Now, that's a broad statement. In terms of more specific ideas about risk, I think for sure cognitive ability tests, as you well know, tend to show a little bit of differential prediction for minorities, so that's something that needs to be looked at very carefully when you're considering those tests. The unfortunate part of that is that those tests have been shown across the board to be the best predictors of performance, because in general there's an understanding that smarter people will generally do better on a job.

Steve

I'm guessing a lot of people probably, I know what a cognitive ability test is, I'm guessing a lot of people listening to this don't. Can you very quickly summarize what are the major types of test people use for selection decisions?

Charles

Sure, absolutely, and you mean within cognitive ability?

Steve

What is a cognitive ability test, how's that different from a personality test? What are the major categories that you're seeing companies might consider using, when they're saying, we need some sort of test to measure or improve our selection decisions?

Charles

Absolutely, well I think cognitive ability is definitely one, and that is a test, it's general, it's how smart you are, how you think about things, how your mind works, and there's a variety of the different types of test that fall under that umbrella, you can think of something like reading comprehension, numerical comprehension, spatial ability, your typical, I'm trying to think of the Watson Glaser, that's a very common one where you're really measuring comprehension and ability to be able to …

Steve

Well they're most similar to the SATs or the college entry exams, probably is the one that most people would be familiar with.

Charles

Exactly.

Steve

OK, so what about a personality test, is it like a Myers Briggs? I know that's one that shouldn't be used for selection, but can you tell us a little bit about what a personality test is?

Charles

Sure, a personality tests really looks at stable constructs, well relatively stable constructs, that you have, the most common one is what we call 'the big five', so you have basic attributes such as extroversion and conscientiousness, openness to new things, things that are part of who an individual is that will then impact how they interact with others in the workplace. Personality tests are something that have been used for a while now, and we're moving more and more to those being much more business-related, and when they first started out, they were much more clinical in nature, and they asked some questions that I think were potentially unsavory, they were very long in length, they're not always the favorite type of tests for applicants to take, and that's one thing that people should definitely consider when they're thinking about using those tests.

They do work, and they are able to show, really I think the best way that they work is to really show some red flags about people, for instance, conscientiousness is a really good one, that is, "Are you going to show up for work? Are you going to steal? Are you going to do the things in the workplace that are indicative of someone who cares about their job and treats others well?"— that kind of stuff, so red flags on those kind of things can be something that are a very good way to have a piece of data that helps you understand why someone might not be the best fit for a particular job.

Those tests, I think everybody's run across those. The Myers Briggs is definitely a good example of one of those, and you're right, I don't really recommend that be used for selection purposes , neither does my dog apparently! So anyway, that one is a standard that tends to get thrown around a lot.

Steve

So in addition to personality tests, there's test of knowledge, which are, what have you actually learned— is that what that is?

Charles

Yeah, and I think a really good example for those would be something like computer programing languages, you apply for a job as an IT professional, and you have on your resume, "I know this, this and this different programing language", it's a really good way to verify, there's often many different types of test that one can take that really measure to some extent their ability to be able to actually know that language, and that allows you to know, OK, this person is not fudging on their resume, they have this general level of competence with this particular body of knowledge.

So those are pretty common in certain areas. You also have, as we talked about before, what we call 'biographical history data', or biodata, which is generally just questions about what someone's done in the past, and those have been usually shown through a process of research to be able to predict certain outcomes down the line, and those are often found in application blanks where earlier in the process of hiring.

Another one that you see a lot of lately is situational judgement assessments, which are where you put people in scenarios. These are really good for customer service, really good for different business types of white collar jobs as well, where you give someone a scenario— "You're working on a team with these three other people, and one person takes credit for your work— what do you do, how do you solve the situation?"— and you're given some response options there, and those are really even beginning now to feed into the creation of more video-based or simulation type things, where you actually see actors or avatars acting out those situations, and you are able to then chime in on how you would react in that situation. So those are really good, because they're scenario-based, and again when you talk about realism and relationship to what it takes to do a job, something like that is often a much more realistic and much more acceptable to an applicant than just asking them a bunch of questions that relate to their personality, which may seem kind of obtuse, and a lot of times people may say, "Well what does this have to do with the job? Why are they asking me if, when I'm in a social situation, I'm usually the life of the party?"— something like that, that's kind of a problem.

Steve

So what I'm hearing is, there's a lot of different tests, and I know, obviously I've worked in this arena as well, there's a lot of different kinds of tests. For somebody who's not worked in this, is there some place that they can get some sort of sense of all these different tests?— because it seems like it's overwhelming— "I'm a small company, I want to use tests to improve my hiring decisions, I don't have the money to engage in a big consulting organization"— what would be a good first step for a company like that, if they wanted to really explore using tests more?

Charles

Books or topics that will, my company website has a lot of good information to get people started, the Society for Industrial Organizational Psychology has some good resources as well, so I would say, start by looking on the web, and looking at various publications and information sources that can provide you with some clear understanding of this. It isn't an easy thing to just pick up one day and master and become an expert at, there's no doubt about that, and that's been something honestly that's been somewhat of a barrier towards the adoption of tests and assessments in general, so it's a real phenomenon, but it's not anything that is that difficult; it's not rocket science, you could say that.

Steve

Despite the name?

Charles

Yeah exactly, so it can be understood after a while, and I think just, you don't have to necessarily engage anyone in a huge giant undertaking to learn about it, I've worked with plenty of people in my client organizations before who have not had any background in testing and assessment, but have been tapped to run an assessment program, and they have educated themselves through both practice, and also taken their own initiative to really look out there at resources.

Steve

OK, one question— what about if somebody's approached, and I know there's a lot of companies that sell different tests, if you're a staffing director, and a company comes up and says, "Hey, here we've got this great test— use it"— what questions should somebody have when they're considering looking at a test? What would be, are there two or three questions that you should ask every vendor, if they're trying to sell you a selection test?

Charles

Sure, I think there are definitely are, one of the first things to do, and this isn't a question, but it's a listening thing— if someone who is selling you a test or assessment begins that conversation, or even in that initial conversation, starts talking about the fact that their test can predict performance accurately, 75% of the time or 80% of the time or 90% of the time, or what have you, right away that's a red flag, just as we know these things are not perfect, they add a ton of value and they're great, but at the end of the day you're working on predicting human performance, which is not something that's been really easy to lock down all the time, so that's the first red flag that I would look for. Beyond that, I really will ask people, "What is the origin of this test or assessment? Where did it come from, what's some documentation you can show me?" Any good assessment should have a technical report or technical manual to go with it, and the organization who's selling it should have an industrial psychologist or psychometrician or someone with that kind of experience and skill set to be able to help talk to you a little bit about, OK, here's how we developed this thing, here's the theory that is behind it, and here's the results we've seen for it, so that's the first thing.

The next thing I would look for is definitely some references for people who are in similar industries or positions as you would be, and want to talk to them about how they, the experience they've had with the test or assessment, and I would also really take a good, strong look at some of the parameters around it in terms of how long is it, what do the questions look like on there, put yourself in the shoes of a job applicant, take the test, look at the test and then look at the report from the test, and in terms of taking the test and the whole testing experience, it's really important to think about it from the standpoint of a job applicant, because those are your customers, they're who are going to be looking at this thing, and you really want to make sure that it's not something that, it's going to shock them or make them feel uncomfortable.

The second part of that is, looking at the report— put yourself in the shoes of a hiring manager, a line manager, whomever, and look at the output from this thing, and see, is it a bunch of, I hate to use the word or term, 'psychobabble', a lot of people like that one, but you get reports sometimes from testing companies, it's typical in past situations, we're kind of moving away from it, but you'll get a tome, like a 50 page report, about this person and how they interact with all the different people, or in different situations, in your office. It's sometimes very unusable, so what I look for in most situations is a very clear, easy to understand report that basically will allow me to quickly get a read on the data that's coming out of the assessment process.

I think the final thing that I really look at is, how do you make sure that the test that you're going to be giving me is job related? What are the things that you do as a company to help me have that peace of mind, because as we talked about before, when looking at legal issues, that's really the number one thing that you have to look out for, it's making sure you can draw clear links between test content and the performance of the job in question, and your company, your testing company is the one that should really be able to help you to be able to understand that, and to be able to implement it, starting with that very important piece.

Steve

So if I just try to summarize quickly the points you've made, really, one: is the company realistic about what tests can do, as you pointed out, these things aren't perfect, and so if they're humble and conservative in the capabilities of the test, they will improve your hiring decisions, but it won't make them perfect; that they've got people in the organization that are specialists in test design and industrial organizational psychology, or psychometricians, so people that specialize in test design, then you talked about the need to make sure that companies similar to yours, or similar types of jobs, have used this test, tried to get examples, then actually look at the test, say how are applicants going to react to this test, how are managers going to react to using this report, then the last thing is saying, what evidence do they have that shows this clearly is related to jobs, so quite a lot of things to look at, but I don't think it's overwhelming, as long as you're focused on that.

The last question that I ask is, where do you see this industry going? You talked a little bit about greater use of simulations— are we going to a world where, before you apply for a job, you'll go work there through the "World of Workcraft" for a week?

Charles

Oh man, I hope not! I definitely see, I think that's a way in the future trend, although things are starting to happen in that direction now, and that's a personal action area for me, but I think in the shorter term, what I'm seeing right now is two different things that I feel like are happening— assessment is continuing to be adopted, there's tons of options out there for people, you're right, it can be quite overwhelming, but it's been made a little bit easier in some ways in that, as we move to talent management platforms and things, such as SuccessFactors offers, and you're getting these bigger packages that allow companies to run their processes, and also provide content related to those processes, assessment is starting to get folded in there, I'm starting to use the term, "embedded assessment", where the assessment just becomes one piece of a lot of other applications, services, functionality, what have you, that are related to, in this case the hiring process, so that is something. It's becoming a little bit more transparent, and it's almost more like, the company that's offering you this stuff, it's got a module in there that allows it to work, and we're leveraging a lot of data and experience to help allow that to be an effective method of assessment. I think ten years ago that wouldn't have been the case, so that's one trend I'm seeing.

Another trend I'm seeing is again, based on the ability for people to harvest a lot of data, seeing a lot more stuff for the smaller and middle market, where companies are actually starting to sell their tests transactionally. An example would be a restaurant associate, front of house restaurant associate, a company has got whatever, 25 years of data on the different values, personality, whatever, that work for those, they're able to now sell a test for that transactionally over the web, where you can just buy one or two of those tests, and then it's making it more affordable for smaller companies. Of course, there's a bunch of cautions that go along with that, but I still see that trend developing, so more accessibility in the middle and small markets for sure, and then more, OK here's an assessment, we may never even have thought that we wanted to adopt it, but we're buying this other package and it comes with it, or it's one option that we can consider, and the people who are helping us implement this are going to walk us through how we can set that up.

So those are the things that I'm seeing right now. Where I think the real dividing line in terms of a quantum leap forward with this stuff is going to come, it's really more with different modalities of assessment, so instead of the typical experience, which really hasn't changed for an applicant, other than it's now on the web, you're filling in radio buttons, you're clicking it with your mouse, you used to be pencilling it in with a number two pencil, but it's really the same thing, you're indicating a choice from a series of answer choices and a standard question that starts it all out.

So we haven't really moved super far away from that, and I think that's really the next big step, and I think it will come more with simulations and more interactive things where the actual work environment can be simulated, and you can ask someone to interact within that environment, and collect information about the choices they make. One really good example of that would be call center stuff, where a call center terminal that someone uses is very easy to replicate, so simulations for call centers are probably one of the more further along.

Steve

So we're starting to see simulations, but as you said, that's a huge leap, and that's probably a whole other conversation. Well Charles, thank you so much for spending time with People Performance Radio, learning more about assessments. We'll have a link to Rocket-Hire's website on our podcast, where there's a lot more information about assessments, and Charles, thank you very much for appearing on People Performance Radio.

Charles

Thanks a whole lot, I really appreciate it, it was very enjoyable.

Speaker

If you would like to be a guest on the show, or sponsor, please drop us a line at podcast@successfactors.com, or you can leave us a message at 650-425-7474. This podcast is copywritten by SuccessFactors. The views expressed are the individual’s own, and do not necessarily represent those of SuccessFactors, SuccessFactors’ partners or customers. See you next week.