Kind: captions
Language: en

Understanding Personality 
Testing with Dr. Clinton Kelly 

Welcome to Testing, Testing 
1, 2, 3, a podcast brought to you by TestGenius. 

Jenny: Welcome everybody. My name is Jenny Arnez. 
I'm from TestGenius, and we're so glad you tuned  

in today, or perhaps you're listening. And 
today we have Mike Callen with us. He's the  

VP of Products from TestGenius. We also have 
Dr. Clinton Kelly from IoPredict. Clinton,  

you want to tell us a little bit about yourself?
Clinton: Yeah. Thanks for having me. I'd be happy  

to tell you a little bit about me. I have a 
background in industrial and organizational  

psychology. And so for those who aren't 
aware of that, I do not work with the  

depressed people at work. That's the question 
I most often get from those who don't do this.  

I think I'm a counselor for people.
Clinton: In the workforce, but no,  

I actually help people help companies decide 
who to hire and who to promote and within  

organizations. So industrial organizational 
psychologists, we focus on improving  

organizations, making things more 
efficient. I specifically focus on  

hiring within organizations. And that's where 
I've spent the almost last 20 years of my  

career doing is making tests for organizations.
Clinton: Whether those be personality tests,  

cognitive ability tests, multiple physical 
ability, interview questions. And so that's  

what I do is for all sorts of different types 
of jobs and different types of organizations,  

I help them make tests to decide who to hire.
Jenny: Mike, why don't you say a couple of words,  

let people know who you are as well.
Mike: Sure. , I'm Mike Callen. I'm the VP of  

Products at Biddle Consulting Group. Our product 
is TestGenius. It's a suite, a hiring suite,  

which is a testing platform. And it contains a 
series of off the shelf office skills testing,  

as well as a program called CritiCall, which 
is for police, fire, EMS, 911, and utility. 

Mike: Dispatchers, call takers, those 
kinds of products. And then we have an  

item banking testing platform system, 
which IoPredict Clinton's company actually uses  

for some test administration. And I would add 
that Clinton and his partner, Jason, at IoPredict,  

worked with us for several years here at Biddle 
Consulting Group and continue to work with us  

here at Biddle Consulting Group, yet they have 
gone on and launched their own company. And we're  

collaborating quite a bit as we continue forward 
in our own direction. Great collaborations. 

Jenny: Yeah, for sure. So today we're going 
to focus on personality testing. And so I  

have to tell you when I think of personality 
testing, because I'm not trained in industrial  

organizational psychology, I'm new to this field.
Jenny: Honestly, it wasn't until I joined Biddle  

Consulting Group that I even knew 
that I was a job like yours existed,  

so I'm new when I think of personality testing. 
If I were to go on to Google and  

in fact, I did that this morning. I typed in 
personality testing, things like DISC profile  

and Enneagram and Myers Briggs showed up.
Jenny: How does that type of personality  

testing connect to the type of personality 
testing that one might do in the workplace,  

that they want to use a personality test to 
hire or to select the best person for the job? 

Clinton: Yeah, that's a great question. 
And there are some similarities, but there  

are important differences between those.
Clinton: They are trying to measure traits  

or preferences of individuals. Some personality 
components, however, tests like the Myers Briggs,  

for example, it was not developed with the purpose 
of making hiring decisions. In fact, the creators  

of the Myers Briggs, they specifically say 
on their website, our test is not designed  

to be used to help hire individuals.
Clinton: It's not designed to predict  

performance in a job. It's more designed to 
give feedback for the individuals,  

maybe preferences of careers they may enjoy, but 
it's not designed to be used. In an employment  

context, neither is like the disc or strengths 
finder. Some of those, they weren't designed  

with employee prediction with hiring in mind.
Jenny: Okay. All right. It's interesting because  

one of my daughters recently applied for a job and 
they asked in the online application, what's your  

Strength Finders profile? What Enneagram are you?
Clinton: Yeah. I actually this morning I Googled  

Myers Briggs selection and I found 
a number of websites explaining how  

to use the Myers Briggs within selection.
Clinton: So people use it, but like I said,  

Myers Briggs themselves says, don't use 
it for selection, but people do it anyway. 

Mike: That's very interesting, isn't it? We're 
really focused on doing things the right way,  

following the Uniform Guidelines. 
And there's, it seems like there's  

a lot of mavericks out in the selection world.
Mike: At any rate, it's just very interesting.  

I'm actually surprised I didn't realize 
that there were people that were asking for this  

kind of a profile. I would imagine that, in terms 
of defensibility, that's a tough one to defend,  

right? If somebody says, what's your DISC profile 
or something like that and you don't get hired.  

If I didn't get hired, I might challenge that.
Clinton: Yeah, it can be problematic. Like I said,  

because when we talk about, tests and validity - 
often misunderstood topics – often those tests are  

not valid for that intended purpose. And for the 
purpose of selection, and yeah, you can definitely  

have some problems if it were challenged.
Jenny: Okay, so a couple of questions come to  

mind. Mike, you mentioned Uniform Guidelines for 
those who are watching or listening to this and  

have no idea what the Uniform Guidelines 
are. Can you guys give a definition? 

Mike: Yeah, Clinton, go ahead. It's your 
space. So I would I would get close,  

but I think you give it the best run.
Clinton: Okay. Yeah, no, sure. I'd be  

happy to. So the Uniform Guidelines were 
created in the late 1970s. So they've been  

around for a while and they are 
the Uniform Guidelines on employee selection  

procedures. And they lay out the, like it says, 
the guidelines or the requirements. If you're  

going to use tests in a hiring situation on what 
you need to do to demonstrate that those tests  

are valid for use in that specific situation.
Clinton: And the Uniform Guidelines when they,  

when we use the term test, that is, it's very 
broad. It's not test in the traditional sense,  

maybe most people think of the multiple-choice 
test. It’s anytime you are reducing your applicant  

pool. So if two people apply to the job and we 
say yes to one person and no to another, whatever  

we did to make that decision, that was a test.
Clinton: An interview is a test, a personality  

test is a test. If you're doing resume screens 
and you make a pile of yes and a pile of no,  

however you, whatever the criteria used to make 
that decision, it is a test under the Uniform  

Guidelines. And so the Uniform Guidelines lay out, 
they lay out guidelines and say, if you're going,  

whenever you're going to reduce 
your number of job applicants to make a hiring  

decision, there are certain requirements 
that you need to comply with. If you don't,  

you can't - you could have problems legally.
Mike: That goes back even to your job posting  

and your choice of where you put your job 
posting, right? You put your job posting in  

a spot where you're not going to get a diverse 
applicant pool. You have limited, essentially  

limited your selectability in that case.
Mike: If you put invalid criteria in there,  

inother words, if you're posting a job and 
it says a driver's license is required,  

but a driver's license isn't really required 
for that job, you have reduced your population  

there as well. And so there's a lot of things in 
our arena that go well beyond the traditional,  

written multiple choice type of test or 
work sample test or personality test. 

Clinton: Yeah, good, great points.
Mike: And I think another thing that's  

important to bring up is that I was taught early 
on that it's not just hiring that is  

selection, it's hiring, it's promotion, it's 
training opportunities that may be available  

to some and not available to others, like in 
a union environment.If you're going to go,  

up to a journeyman level from, I forgot what the 
basic term is, but those are all selection, right? 

Clinton: Yeah, you're getting put into what we 
call maybe a high potential group. Sometimes  

organizations will identify high potentials that 
are then put on like a track for management track  

that inherently comes with potential increased 
earnings, other things that are benefit that  

tie into selection.So whatever you're using to 
identify like high potential,those could also  

fall under the Uniform Guidelines requirements.
Mike: Interesting. That's great. Thank you. 

Jenny: So is there a separate definition 
for personality tests that's unique,  

that's specific for what you do?
Clinton: Not so much a definition.  

I would say more that intended [00:09:00] use of 
personality tests for selection.They are built and  

designed for use in selection. So that is their 
intended purpose. The way I like to compare it as  

just like a test in general. Let’s just say I have 
an accounting test used to hire accountants and  

it's a great test to hire accountants. It is valid 
to hire accountants. And now let's say I give that  

test to help me decide who I should pick in my NBA 
draft in my NBA lottery, who I should pick next.  

And I give, my NBA draft prospects this accounting 
test. You'd probably say, what are you doing? That  

test is not valid. It's not that it's not a valid 
test. It's valid for helping me hire accountants.  

It's not valid to help me hire NBA players. And 
so that's similar to like these different types  

of personality tests. It's not that they're not 
valid like Myers Briggs. It's just not valid to  

use in a hiring situation. Yeah, for that purpose.
Jenny: So you've used that word valid a few times  

now. Do you want to give a definition 
of what that actually means?

Clinton: Yeah. And so validity, 
there are different, what we call  

types of validity or evidences of validity.
Clinton: And what the most common one for  

personality is criterion, what we call criterion 
validity. To not go into lots of different types  

of validity, because I don't think this 
is the purpose of today's conversation,  

we'll briefly cover a personality 
test under the Uniform Guidelines.  

It requires criterion related validation.
Clinton: So if you are using a personality test  

for selection, there should be some evidence of 
criterion related validation. And if we pull up,  

I can pull up a slide here. We can show 
a few here, criterion validity, what that  

is a mathematical relationship between how people 
score on the And some measure of job performance. 

Clinton: So for example, if we say, higher scores 
on the test, they're more likely to sell more of  

the product if it's a sales position. So if 
we say higher scores on the test, they make  

more sales that would be, if we can 
mathematically show the relationship between  

your score on the personality test and how much 
product you sell, that is criterion validation. 

Jenny: Okay.
Clinton: And this is a mathematical  

relation to show this kind of graph it out. You 
can see if we plot here on the X axis, we have  

test scores that range in this case, 0 to 100. 
And we have some sort of measure of performance on  

the side. We can graph this. You see this person 
has a test score of 22, a job performance of 31. 

Clinton: Over here, test score of 85, job 
performance of 55. You can visually see the  

relationship amongst these dots. What criterion 
validity is it's you're coming up with, we call  

it validity coefficient, which is essentially just 
a correlation coefficient in many cases that shows  

this linear relationship between how you score 
on the test and how you perform on the job. 

Clinton: And so that's what we mean by validity 
for personalities. We can show that this is a  

valid predictor of [00:12:00] success in the job.
Mike: So that validity coefficient would be like  

the strength of the validity, right?
Clinton: Yes, the strength and the  

higher that validity coefficient if it's 
a traditional correlation coefficient,  

it's going to range from 0 to 1 with 1.
Clinton: 0 being a perfect correlation,  

which that doesn't exist. But in theory, 
you could get there. The higher that number,  

the stronger that relationship visually, 
what that would look like a correlation  

of 1. Every blue dot here would be 
perfectly on this red line. That would  

be a correlation of one. So the tighter 
these blue dots are to this red line,  

the stronger that correlation coefficient.
Clinton: The more spread out they are around  

the line, the lower that value is going to be. 
The closer it's going to be to zero. Perfect.  

Correlation coefficient of zero would essentially 
be saying, it doesn't matter how you score on the  

test, we have no idea how you score on this test 
gives us no indication whatsoever of how you're  

going to perform on the job.
Clinton: That would be a  

correlation coefficient of zero.
Jenny: And where does the criteria come from? 

Clinton: And there are, that is flexible. The 
Uniform Guidelines say they just need to be  

important criteria. And they mentioned some 
examples. Some could be supervisor ratings of  

performance. It could be a sales and a sales job.
Clinton: It could be turnover. So we could  

correlate with turnover like our people. Can 
we predict maybe who's less likely to turn over  

by with this personality test? And so that is 
open and it just needs to be what they call  

criteria that are important to the organization 
to the job. So most typically, it's going to  

be some sort of measure of performance, if it's 
sales supervisor ratings or turnover, those are  

the types of things that are very common to see.
Mike: And, when you have a test like this you can  

assemble this test and then you can just, start 
collecting all sorts of information and find out  

that a test correlates to some sort 
of aspect that isn't necessarily valid. It wasn't  

intended to select for that purpose. So can 
you talk a little bit about that philosophy  

of not just creating a test and throwing 
everything up against the wall to see what  

sticks versus having some intention when you 
design? It's a little bit risky, we've gone  

through this step with you folks many times.
Mike: And there's a great deal of risk when you  

go through and you put this together because 
you could potentially go through and not find  

what you're looking for. And that doesn't open up 
the ability to find, some other unrelated thing, 

Clinton: Yeah, no, you need to be intentional 
when you're designing these tests. 

Clinton: We don't just, like you said, 
just throw a bunch of stuff against the  

wall and see what sticks. And so whenever we're 
designing a test, a personality test for a job,  

we want to be intentional and we want to 
do what we call a job analysis where we  

analyze. What is done on the job? What are 
the requirements of the position?

Clinton: What are individuals spending their 
time doing? What are the most important parts,  

the most difficult parts? We're going to 
interview employees, talk with them. And  

so you need to do your research and be intentional 
with what you are doing. One of the, I'll share a  

slide here. Sorry, I'm going to skip forward.
Clinton: This is an example for one we put  

together for a factory worker. And this doesn't 
show everything. We did multiple interviews and  

we even went on site and watched employees, but 
we took a look and we said, Hey, there's some  

attributes we've noticed in our research, in our 
analysis of the job, the workers that appear to,  

in this case, stay on the job, here's 
some characteristics that we're seeing. 

Clinton: These eight things that they're 
self confident, analytical, forthright,  

and et cetera. It takes some time. So 
you need, like Mike, like you said,  

we need to be intentional and purposeful. 
So you're not just saying let's just throw  

some stuff out there and see if we get lucky.
Mike: So we, when we've created personality  

tests we've worked together to create
personality tests, we have used a concurrent  

validation strategy. And I know that, basically 
I would say that, from my perspective, the two  

basic ways to do to validate a personality test 
would be either a concurrent study or a predictive  

study. Why don't you talk a little bit about that 
and, maybe what some advantages of either are. 

Clinton: Okay. So great. So those are the two 
types, the concurrent or predictive. Concurrent,  

what that is done typically with your existing 
employees. So if you are in a job and you have,  

a few hundred employees already employed in this 
position, what we do is we develop the test or  

take an existing test from some vendor.
Clinton: We administer it to your current  

employees. And then we correlate that with 
measures of job performance. So if you have  

existing measures to job performance, or we 
can create new measures of job performance.  

And so that is a concurrent one. As you take your 
existing employees, have them complete  

the test and we correlate it with job performance.
Clinton: Predictive is where we start to give  

the test to job applicants. And often you'll 
not be scoring, it's collecting data in the  

background. So people are applying for this 
job and you're giving 'em this new test. The  

applicants don't know that it's not being used, 
but you're really not considering their score. 

Clinton: And as you hire new individuals on 
the job you then, at a later point in time,  

correlate how they scored on the test with 
measures of job performance. And so both  

you're trying to do the same thing- correlate test 
scores with job performance. The main difference  

is concurrent is with existing employees.
Clinton: Predictive, usually use it with job  

applicants and at a later date, you have to come 
back. So the advantage, you can probably tell just  

from the way I'm describing it, the advantage of 
concurrent is it's typically faster. You can take  

your existing employees. You don't have to wait 
six months or a year to find how they panned out. 

Mike: Yeah, and your population is much more 
under control as well, because obviously,  

if you're doing a predictive study,
you're going to be testing all these applicants,  

and you might not hire 75 percent of them. So 
you've actually collected data that you're not  

then going to be able to apply later. And 
then I guess one so one advantage of the  

predictive study is that you get the broadest 
range of people that in terms of knowledge,  

skill, ability, and personal characteristics 
with the concurrent study, you have a little  

bit of range restriction. Is that not correct?
Clinton: Yeah, we can't, you can get some range  

restriction because these individuals are current 
employees and we assume they're performing at an  

adequate level or else they would not be current 
employees or they would no longer be employed  

there. So that can be a downside is that range 
restriction is because there are scores on your  

test. There's maybe not as much variability 
or differences in how they score. One of the,  

one of the upsides of a predictive 
study is you are getting, like you said,  

the range of responses and you can also 
get, you're getting that real  

life situation from the job applicants.
Clinton: They are really applying for a  

job. So you're mimicking what you intend the 
test to be used for in a concurrent sample.  

When you're giving it to current employees, 
they already have the job. And so they may  

complete the test from a different frame of 
reference than your job applicants. And so  

that can be a potential downside of a concurrent.
Clinton: And so there are benefits and negatives  

to both approaches, but both are equally 
allowed under the uniform guidelines. And  

both have been shown from research perspective 
to, to basically result in creating tests  

that are predictive of performance..
Jenny: So you're mentioning developing  

tests or validating tests that 
are used for employee selection. 

Jenny: Are they ever use it 
a post hire environment for  

perhaps employee development or training?
Clinton: They can be, but that's often what  

you'll see, maybe with Myers Briggs or DISC 
or Strengths Finder some of those. And so  

for the intended use, again, it comes back to 
that intended use. Depends on what  

the test was designed to be used for.
Clinton: Some tests have multiple,  

have been designed to be used for multiple types 
of things. But this is something to be aware of.  

What is your test designed to do? And sometimes 
I'll see this with clients who want to take a  

test. And not necessarily a personality test, but 
let's say a technical skills test and they want  

to use it in a diagnostic way to diagnose where 
someone is needs training or things like that. 

Clinton: And I'm like, that test wasn't designed 
to do that. It gave you some initial information  

to make a hiring decision, but it really wasn't 
designed to diagnose and tell you here's their  

training needs. And so you just need to be 
aware of what your tests can and can't do. 

Mike: There's a a maxim in HR as well, that 
says job performance trumps testing. One of  

the things that happens is when a person goes 
from applicant to candidate, to employee is  

that you start to have these instances where, 
job performance is being recorded.  

And so very often we have people who will 
ask us, Oh, is it okay to give our employees  

these pre-employment tests and then, if they 
can't pass them any longer to get rid of them. 

Mike: And generally speaking, that's a really 
dangerous area to tread into because, you have a  

much better job-related measure of what's going 
on, which is their performance on the job at  

that point. So at any rate, it's just an aside.
Clinton: That's that's a great point and kind of  

shows in this, you can see that in this 
graph, tests are by no means perfect. 

Clinton: Like you said, job performance trumps, 
trumps the test scores. Like in this line here,  

you can see some of these blue dots, they're 
not on the line. So in some cases, the test,  

a person may out predict what we outperform 
and what we predict. In some cases, a person  

may underperform what we predict.
Mike: And so just to be clear here,  

any one of these dots is the 
intersection between an individual's  

test score and their performance on the job.
Mike: So that one that's in the middle  

top there above the word test score. There's 
somebody who scored approximately 50, but rated  

almost 60 on their performance measure, right?
Clinton: Yeah, so in that case, they performed  

better than we thought they would on the job.
Mike: And then conversely, down below the  

line to the left of the 40,  

there's somebody who scored, is that right?
Mike: Am I saying this right? Yeah. They under  

tested and under performed, either way, right?
Clinton: Yeah, but they were a lower test score,  

but even then, the performance 
was lower than was anticipated. 

Mike: Yeah. Thank you.
Clinton: And so that's  

something that the tests are, but the tests are 
not perfect measures, but what I always say is  

it's, they're better than the alternative.
Clinton: And so they are getting you,  

they are increasing your odds. And so it's like 
when people sometimes will say this person is just  

not a great test taker. I often will say there's 
also sometimes someone's great grandma who smoked  

40 cigarettes a day and lived until she was 97. 
But you probably would not say in, in, in general,  

the trend is if you smoke a lot, you're 
likely going to have some other health issues. 

Mike: We're going to expect everybody who smokes 
several packs a day to live to a hundred. That's  

not the, that's not the case. Another side of 
that's really important, at least I always try  

to bring this up when I have this conversation 
with people, is that the alternative is that  

a human being is making a selection based 
upon, in terms of the applicant, nothing. 

Mike: You have selected somebody who will be 
hired and you have selected somebody who will  

not be hired, for apparently no reason 
whatsoever, or no concrete reason. And  

so that's another reason why to have this 
decision making process tied to something  

that's observable that you can find patterns in.
Mike: Or if you can find that there's disparities,  

you can measure those things and see what 
exists and then remediate that situation. 

Jenny: Yes, so if an employer would like 
to begin using a personality  

assessment. How do they begin?
Clinton: There's a, I guess there's  

a couple of different ways. There are, 
you can, reach out to vendors out there. 

Clinton: There are two different approaches. 
There are, is what I would call the build a  

custom test from scratch approach, and there 
is the off the shelf approach. There are  

existing personality tests that are off the 
shelf. And then there's the, Hey, we build  

it custom for this particular job approach.
Clinton: What you're looking for may depend on  

your situation in your organization. If you're 
a smaller organization and, or it's a role in  

your organization where you don't have that many 
employees. Off the shelf is probably going to be  

the better route where you, a vendor has spent 
the time to develop a test for that type of job. 

Clinton: And because you just don't have to 
do a sort of study where you can show this  

mathematical relationship between test scores and 
job performance, that requires data. And if you  

only have 10 employees, Iin a certain position 
in the organization, you don't have the data and  

the necessary data to demonstrate a 
relationship and have any confidence in that. 

Clinton: And so in that situation, you're going 
to need to rely on an off the shelf test where  

they've done some of that research and can show 
how it predicts performance for that type of  

job. If you have a large organization, 
you may be in the spot where you say,  

let's custom develop this. If we have a few 
hundred employees in a position, you can do that. 

Clinton: So those are the kind of the 
2 approaches. And I'm sure there's,  

I can get into like, how can you continue to want 
to get into how we do that once identified maybe  

which one, the custom or the off the shelf.
Jenny: Yeah. Yeah. I think you should. Yeah. 

Clinton: So then when you get 
a customer or off the shelf,  

there are different types out there of tests.
Clinton: A lot of them, when it comes to  

personalities, they're typically statements. When 
you take a personality test, you're often reading  

statements. And I'll show here a couple examples. 
This is an example we'll call a forced choice  

personality test, where you say, Hey, I'm more 
likely to, and this is a silly example  

here, but eat a salad or eat a hamburger.
Clinton: And you have some points in the  

continuum. Sometimes you're just given two 
options. You just pick one or the other.  

This point, you're given four choices on other 
personality tests. You're just given a statement  

and you just slay like true or false, or that's 
like me or not like me. For example, I enjoy  

team projects and you would say true or false.
Clinton: Other times one of the most common still  

is we'll call it like your type skills that 
strongly disagree to strongly agree. Again,  

I enjoy team projects or strongly disagree or 
somewhat disagree or to the strongly agree. So  

personalities generally are in that kind 
of flavor where they, you can see they  

have statements and you're making some sort of 
agreement or disagreement with that statement. 

Clinton: And if you're looking to implement it, 
so you need to go, if you're going to the vendor  

route, go to the vendor and you should ask the 
vendor about what this test was designed for and  

its intended use to make sure they 
built this for selection. And not only built it  

for selection, but they have validity for your job 
type, going back to that accountant and NBA player  

example to make sure they're not giving you a 
test that were designed to select NBA players  

and you're trying to use it to hire accountants. 
Just what was this test? Where, what validity  

evidence do they have for your job type?
Mike: So as well, Clinton, there are many,  

I may not be using the right word, but I would say 
standardized personality tests where the battery  

like Hogan, for instance, they're going to ask the 
same 110 questions of every person, but over time,  

they're going to do small studies with certain 
job titles or certain groups of people in order  

to be able to take this standardized test and key 
it or score it in such a way that it doesn't have  

meets with that particular
population on the other side of the coin. 

Mike: There can be a custom test, which 
wouldn't necessarily have standardized  

questions. They might have questions that are 
written that have more face validity. They're  

more written directly towards this particular 
job title that you're using the test for,  

is that not correct? And do you have any 
recommendations about that, or any positives,  

negatives about either of those scenarios?
Clinton: Yeah, that is correct. Most of the,  

a lot of the tests out there that measure 
what we call the big five personality  

characteristics. And it, like you said, Mike, 
it's the same exact test for every position. 

Clinton: The difference being what, where you 
need to fall on the characteristics. So let's  

say openness to experience. You may, they may 
say on this job profile or job type we found that  

people who are more likely to be successful are 
high in openness to experience. And another job  

type people who are more likely 
to be successful are in the middle range  

on openness to experience for the lower range.
Clinton: And so they're given the same exact test,  

but there's a different, like constellation of 
profiles of where you fall in these big five  

personality characteristics that they're saying 
is indicative of success. And I don't know if  

there's really like a, say this one is better 
than the other, but when you have a test that's  

custom designed for a specific position, the good 
thing about that is that it was, like I said,  

that it was custom tailored for that job type.
Clinton: And so that, again, the frame of  

reference, even the way the items are written from 
what I've seen, just in my personal experience  

with work, I tend to see higher validity 
coefficients with those tests when design,  

when they are well designed and put together. 
Higher validity coefficients than with the  

general tests that are just Hey, we then custom 
create a profile depending on your job type,  

but it's the same exact test for all jobs.
Mike: Yeah. I think another aspect that's  

very important to maybe particularly important 
now in this age of social media is  

the applicant satisfaction part of it. 
When I was growing up and I had nothing  

to do with this particular career, I took 
a lot of tests or did interviews or did  

physical ability tests, those kinds of things.
Mike: And so I've often reflected back on those  

processes and thought about which ones were very 
satisfying to me. They had the face validity and  

face validity is basically, how does it feel? 
Does it feel to the applicant? Like it would be  

something that they might encounter on the job. 
So it doesn't mean the test is valid or not. 

Mike: It just really has to do with my 
own impression of it. And I do remember  

going back through in my mind, some of these 
experiences I had and I was like, Oh, that  

was a valid test. That was not fair. That was a 
very unfair experience. And I think some of these  

personality tests, where you're asking, would you 
rather be, at a a noisy party with loud music or  

walking by yourself on a beach?
Mike: What does that have to do with being in  

ABC job? There's a lot of things that don't feel 
really good about that. Whereas if you're asking  

it in context of the job, do you prefer an active 
work environment where your day goes by quickly  

versus a quiet environment where it's maybe a 
little less stressful or those kinds of things  

that has a lot better feel to it than the other.
Clinton: That's a great point that face validity,  

which is not what we always say. It's not like a 
technical type of validity, but it is real in the  

sense that candidate perceptions of the experience 
matter. If candidates feel a positive experience,  

they're less likely to, for example, to challenge 
a test or beat it up or complain about it. 

Clinton: And that is one good thing of 
the custom develop test too is that they  

tend to have a little more face validity 
because the questions by written with that  

job in mind and so yeah that is one benefit 
another benefit of custom developed tests. 

Mike: Perfect. Thank you.
Jenny: So here's  

what we're going to do.
Jenny: We're going to take  

a short break and then we'll be right 
back with everyone in just a moment. 

Voiceover: And we'll be right back 
after a word from our sponsor. 

Ready to revolutionize your HR strategies? 
Head over to TestGenius.com to discover our  

latest tools and solutions designed 
to streamline your hiring processes. 

Jenny: Welcome back everybody. And just a 
reminder, we're with Dr. Clinton Kelly from  

IoPredict, and we're talking about personality 
testing. So when someone takes a personality  

test or a personality assessment, a score is 
spit out for that person, correct? Can you talk  

a little bit about personality test scoring?
Clinton: Yeah. So when it comes to the scores,  

there are different scores that are done and 
those, when we talk about those off the shelf  

tests that measure the big five personality 
characteristics. You'll often get a scale, a  

score in each of those big five personality, what 
we call sub skills. You'll get like an  

openness to experience score, an extroversion 
score, conscientiousness score, and then you'll  

often get an overall fit recommendation.
Clinton: And this is something that's on  

the screen. This is for a custom developed tests 
that Biddle and TestGenius, where you give like  

an overall recommendation. And you can see here 
on the screen here, in this case, we say, Hey,  

people are highly recommended. They have a 77 
percent chance of being successful on the job. 

Clinton: And this is based on job performance data 
collected from actual individuals. You can see  

the numbers here, the number of individuals. 
And so what we give on the score, we give a,  

usually give an overall recommendation score. Most 
tests you'll see like some sort of overall score.  

And then you'll see some sort of subscale scores.
Clinton: The overall score is typically what is  

going to be the driving factor in whatever 
decisions you are making or informing the  

decision. What you don't want to do, 
and is tempting for many people to do,  

who are not super familiar 
with personality tests, is take subscale  

scores. And let that drive your decision.
Clinton: Because the subscales are a sub  

component. They are sub, like it says, they're 
sub to the overall score. So less data points  

from the test are driving subscale scores 'cause 
it's just a portion of the overall test. And  

so it really should be that overall score or 
profile score you get from a personality that  

should drive your recommendation and those sub 
skills can be informative pieces of information  

to then maybe dive into maybe in an interview or 
another step of the selection process. But don't  

let a subscale score. I've seen this happen where 
someone says there's some overall profiles like  

we highly recommend them based on the overall 
profile, but they'll see a subscale score where  

they're fairly low on some subscales I don't 
know if you should hire this person - I'm like,  

no, don't let that drive the decision.
Mike: That's a really important point  

that you brought up about the subscale is 
that you can use it to drive questions in  

the interview process. In fact, very 
often the reports I know the reports that are in  

the test that we've created together, you've 
included those drill down questions because  

that gives you a really great opportunity 
to, a find out, did the test potentially  

really nail this subscale or did it maybe miss 
the subscale or furthermore is this weakness,  

maybe something that the candidate is familiar 
with and has the ability to work around it, which  

can turn a weakness into a strength. So is that 
always something that's included with personality  

tests for selection or are we just lucky?
Clinton: Yeah, not always. So Yeah,  

they're not always included, so that is, yeah, so 
there are some where they just give you an overall  

score and and I've even seen some that just give 
you an overall score, there are even those sub,  

sub scores, and then, but yeah, so it is just, it 
is nice when you get more information like that,  

there are a number of tests out 
there that will provide sub scores and even  

some additional information or potential 
prodding questions, but yeah, it is,  

like I said, it's great when you can have that 
additional information to prod into an interview. 

Clinton: Yep. But like Mike had said, just 
don't let us, don't let a subscale score drive  

your overall decision. It's like letting, like one 
quarter, one quarter of performance in a business  

drive your overall decision. If you don't look 
at the yearly profit and loss, you're like, no,  

over a year we made a lot of money.But one 
quarter one, it was slow. And now you say,  

Hey, we've got to close the company 
because things are horrible. It's no,  

don't let the subscale score, the overall score.
Mike: I want to tell a quick story because  

this was something that was so 
impactful about 10 years or so ago. 

Mike: We created a testing process for 
nurses and it included knowledge testing,  

video situational judgment testing, and 
personality testing. And one of our.  

Clients told us about a instance where they were 
reviewing the results with the applicant in real  

time. So they had tested and then
they moved them onto an interview process. 

Mike: And the experience level of this applicant 
was very good. The knowledge was very good. The  

situational judgment was good, but there was 
this one subscale, which was hostility. And  

there was a red flag on the hostility scale. And 
the interviewer asked the first prodding question,  

Hey, so tell me about a time, and it was some 
question that had to do with when they were in  

a stressful situation, how they didn't handle it.
Mike: And then they asked the second prodding  

question and then started, started to 
feel that this was exposing something,  

asked the third prodding question, and at this 
point, the candidate slammed her hand down on  

the desk, said, I've been to anger management 
classes. I've worked through this. If you  

would just move on, I would appreciate it.
Mike: There was a great instance where,  

you know, that this did bring up 
something that should have been discussed  

and was discussed and herein is the real 
value in collecting that data and having  

an opportunity to talk about it because, 
boy, when you talk about dodging a bullet,  

that was an important one right there.
Clinton: And that is a great example of how to  

use that. In some cases, those subscale scores, it 
may be you may find something like you said that's  

yeah, this is a red flag, but more, more often 
than not, I'll see individuals without prodding,  

like I said, without doing those follow up 
questions, take a subscale and automatically maybe  

disqualify someone because of a subscale score.
Mike: And we talked a little bit about how you  

use it for selection. There's a, we do a lot of 
work in the 911 space, and I know we're going  

to do a follow up podcast where we talk a little 
bit more about that. But there's a, there's fairly  

popular personality tests in that space that 
are content based validated, which  

the Uniform Guideline specifically says, 
don't content validate a personality  

test.You want a criterion validate it.
Mike: So let's talk, why don't you talk  

a little bit about the defensibility aspect? 
I think that the stuff we've talked about up  

to this point is really the utility aspect of 
the validation. Here's why you go through this  

process so that it will do a good. Good job of 
selecting, but why do you go through this process  

to make sure that if somebody, an applicant or 
the DOL or the DOJ or the EEOC or some plaintiff's  

attorney has a problem with something that you 
did, what, why does this, how does this come  

into play and how does this help somebody ease 
out of an uncomfortable defensibility situation? 

Clinton: Yeah, great question. When it comes 
to those situations, if you have a test that,  

for example, is just content valid or 
you don't have the proper validation  

in place for a personality test to show that 
it's predictive of performance,  

if you end up passing, let's say your personality 
test ends up passing more males than females or  

more females than males or in one or just more 
of one protected group over another, that's  

when the Uniform Guidelines kick into place.
Clinton: The Uniform Guidelines say, Hey, if your  

test screens out a disproportionate amount of one 
protected group status, you need to show that your  

test is a valid test and that it's screening those 
people out because they can't do the job. And it's  

not screening them out just because they're female 
or they're Hispanic or whatever group that may be. 

Clinton: And so if you are using a test that 
doesn't have the proper validation in place,  

and let's say you're screening out a 
disproportionate number of some protected group  

status, you will lose in court if you get sued.
Mike: Yeah. Good point. That's good advice. It  

doesn't happen very often, but it does happen 
that a process gets questioned and it can really

financially impact an organization in 
terms of legal decisions that might go towards the plaintiff,

but also it hurts their reputation in 
the space.

You don't want to be that group that got sued because they did something incorrectly.

Clinton: Yeah. So if you're a vendor out there and you're not doing your due diligence, 
yeah, it can really hurt your reputation. Clinton: And that's what I tell with clients 
and said, yeah, you can, we can put out a test  

that doesn't have the proper kind of validity 
to back it up but I don't want to do that.  

Cause that's going to, that's going to hurt 
my reputation as an individual in the field. 

Mike: Yeah. Yeah. And so you and I know, and 
a lot of people may or may not know that,  

when criterion validity is established elsewhere, 
the Uniform Guidelines in 7B does allow for  

validity to be transported from the environment 
in which it was established over to an employer's  

own specific environment.
Mike: Why don't you talk a little bit  

about what that means? Processes and 
any recommendations that you have. 

Clinton: Yeah. And that is a 
really great strategy for jobs,  

especially where you just don't have the sample 
size to conduct these criteria types of criteria  

and validation studies in your own organization.
Clinton: And so the Uniform guidelines allow us,  

let's say Mike, cause we've done, we'll talk 
about this in a later one, but we've worked with  

the 911 space with personality testing. 
And a lot of 911 dispatching agencies,  

they just don't have hundreds of dispatchers. 
They have maybe a handful. There's a lot  

of smaller jurisdictions out there.
Clinton: So if they want to implement  

a personality test and say, Hey, what profile 
or what type of personality is more likely to  

be successful in this job? They just 
logistically don't have the numbers to  

do such a study. And so what 7b, section 7b of 
the Uniform Guidelines allows is it says, hey,  

if a validation study for let's say 911 
dispatchers on the personalities has been  

conducted with a specific test, you as 
a smaller group, you can adopt this test and not  

have to do your own criterion validation study.
Clinton: If you can show that your dispatchers,  

your 911 dispatchers perform substantially the 
same major work behaviors as those 911 dispatchers  

where this study was originally conducted. So 
basically you have to show, hey, there's a match,  

there's overlap between my job where I want to use 
this test and the job where they, what that they  

used to initially develop and validate this test.
Clinton: And if you can show that the two jobs  

are substantially similar in the tasks 
and the work behaviors they perform,  

you can transport that validity 
under Section 7B of the Uniform  

Guidelines. And that fulfills 
their validation requirements. 

Mike: My understanding is that when you're 
looking at these work behaviors that were  

covered under the initial study, that's 
the work behaviors that you're looking  

at in terms of transporting this validity.
Mike: There may be some other work behaviors  

that weren't covered under that validation 
study that aren't really a part  

of this conversation. But in terms of 
being able to transport the validity  

from this original study and bringing it over 
to your environment, is it not those particular  

performance dimensions that were purported to be 
measured or in that original study by that test? 

Clinton: yeah, you do want to look at those. 
So those original things that are measuring  

that test are those work behaviors also 
important in your job? And and you say,  

that's the critical part is showing that overlap. 
And it's not, it is not super difficult to do it  

is as long as I say as long as in the original 
study, the vendor did their due diligence and how  

they showed what the work behaviors were in that 
original study. So if they've done their homework,  

it's a very easy process. I have seen this in 
the past where a client that I was working with  

was wanting to use a test.
Clinton: And they had some  

criteria and validation, but they weren't able to 
give me information on what the people in those  

jobs did or what exactly they did in that 
original study. And so it was,  

we couldn't really transport the validity because 
they didn't do their homework on that initial  

study. And so that's why it's important.
Clinton: When you do that initial study,  

the vendors should have done a good 
thorough job analysis of analyzing  

the job. So you can make sure there's a 
match between where they did the study and  

your job where you want to implement the test.
Mike: That's a really great point. And we might  

want to toot our horns a little bit right here.
Mike: But, being a shop, both of our shops  

are shops where we lean very heavily on the 
Uniform Guidelines and, the principles and  

the legislation and, laws that have been put 
into place so that we are doing things right  

to begin with so that if somebody does determine 
that they'd like to transport that validity over  

to their environment, they're able to do that.
Mike: But you do really want to do your research  

or as much of the research as you can to make sure 
that you are partnering up with an organization  

that has the same values and is doing 
things right to begin with. Otherwise it's the  

garbage in garbage out philosophy, right? If 
you start to transport over a validation study,  

that's a bad validation study.Then, 
you're protected by nothing, right? 

Clinton: Yeah. And so that's one thing. If you're 
looking to implement a study is ask that vendor  

to say, Hey, do you have a validation study? They 
should have a technical report. If you're asking  

a vendor who's got a personality test and you 
ask questions about validation or do you have a  

technical report that documents things and they 
start to kind of him hot or dodge you or they  

send you like a one page more a marketing flyer.
Clinton: Those would be red flags in my opinion  

to say, this person may not have done their 
homework. You should be hearing things like  

Uniform Guidelines, like criterion 
validation. There's certain terms,  

keywords that the vendor should be mentioning.
Mike: Very good. We've mentioned Uniform  

Guidelines a lot of times.
Mike: So we actually have a  

website set up. That's uniformguidelines.com where 
the Uniform Guidelines are in there.  

Topically by heading hyperlinked. There's also the 
Questions and Answers to the Uniform Guidelines,  

which came what, 10 or 20 years later, right? Or 
I don't remember how many years, but that was,  

they circled back through and said, okay, we've 
got this document, looking back now upon it,  

what are different scenarios that have been 
encountered and how did this get interpreted? 

Mike: So both the Uniform Guidelines and the 
questions and answers are very valuable in  

terms of knowing what the best practices are for 
testing and selection. So uniform guidelines.com,  

you can go there and you can, take a look at it, 
bookmark it and refer to it anytime that you'd  

like, that's a resource that we sponsor.
Mike: Jenny, what else? 

Jenny: I actually think we're getting to a point 
where we're going to start to wind down. Okay. And  

just want to let everybody know this is part 
one or episode, session one with Dr. Kelly.  

And next time when we meet, we're going to be 
talking about, specifically about personality  

testing in our CritiCall application.
Mike: Okay. Which would be the dispatch realm. 

Jenny: 911 emergency dispatchers. So let 
me do this. Let me change the screen here,  

bring us all back. And as we wind down Clinton, 
if someone wants to get in touch with you,  

how do they do that?
Clinton: Yeah. So you  

can reach out through our website at iopredict.com
Clinton: And my email is just CKelly. So my name,  

first initial, last name, CKelly at ioPredict.com. 
You can also reach out to Mike over at TestGenius  

and Biddle. And like I said, we have a great 
relationship and we work together hand on hand  

and hand on a lot of validation projects. 
And so those are some ways to get in touch. 

Mike: And you and Jason still are maintained 
at Ckelly at biddle.com as well where we work  

together very closely and we continue to 
really value this relationship and want to  

continue to be able to collaborate
together for many years to come. It's been a,  

it's been really great. And I for one, would 
like to thank you for being on with us today. 

Mike: , I think we maybe even went a little bit 
longer than we had targeted. But it's been really  

valuable hearing from your experience regarding 
this subject, which is, can be a little nebulous,  

it can be a little scary to tiptoe into. 
So I've learned a lot and thank you so  

much for your time today.
Clinton: Yeah. Thank you  

for having me. I really I enjoyed it.
Jenny: Oh, it's been great. And we do have  

show notes that will show on the page where you 
see this video. We'll have links to IoPredict's  

website to uniform guidelines.com and other 
resources that were mentioned in this video. So  

thanks to everyone for listening and for watching.
Mike: Great. Thank you. 

Thanks for tuning in to Testing, 
Testing 123 brought to you by TestGenius and  

Biddle Consulting Group. Visit our website at 
testgenius.com for more information.