Podcast-Intro: Welcome to
Testing, Testing 1-2-3, a podcast

brought to you by TestGenius.

Jenny Arnez: Welcome everybody.

My name is Jenny Arnez.

I'm your host for today's podcast.

I have two gentlemen with me today.

I have Mike Callen.

He's our VP of products at TestGenius.

I'm also part of the
TestGenius team and we have Dr.

Chris Cunningham.

He's the chief science
officer from Logi-Serve.

Mike, you want to go ahead
and introduce yourself?

Tell us a few things.

Yeah.

Mike Callen: Hi, thank you, Jenny.

I'm, I'm Mike Callen, as you
said, and I'm the VP of products

at, at TestGenius, which is a
product of Biddle Consulting Group.

And TestGenius is a skill and ability
testing platform that handles a lot of

different Positions and we have been
mostly engaged in doing concrete, hard

skills testing over the years and through
an integration with Logi-Serve with

whom for whom Chris Cunningham works.

We now have the ability to add a
really substantial library of soft

skills tests or behavioral tests.

And so with that, we want to turn it
over to Chris to introduce himself.

Chris Cunningham, PhD: Yeah, thanks guys.

I appreciate that.

Yeah.

So I'm, I'm Chris Cunningham.

I'm the Chief Science
Officer at Logi-Serve.

And what that means is I use a background
in industrial and organizational

psychology to guide the development
of our assessments and the ongoing

validation and management of those
different testing libraries the way

that they're applied with our clients.

And as Mike alluded to we are
focused in more of a soft skill

space with these assessments.

We tend to focus on competencies,
which you can think of as

complex individual differences.

They're more than just traits,
but they're not states.

So what that means is they're fairly
stable within individuals, but they

can be developed with concerted effort,
focused effort, and these particular

competencies that we target tend to
focus on behaviors associated with a

person's underlying service orientation.

And that's one of the reasons why this
partnership is potentially so powerful.

Jenny Arnez: Wonderful.

Mike Callen: Absolutely.

Jenny Arnez: Chris, can you tell
us a little bit about Logi-Serve?

Chris Cunningham, PhD:
Yeah, I'd be happy to.

And so again, just for the visual
learners out there here's, here's a

bit of a representation of some of the
things we've been up to as a company.

We've been around for
a little over a decade.

We are a provider of talent management and
assessment products that we have developed

in house and are purpose built to help
organizations take a more competency

focused approach to their screening
and hiring and developing of talent.

As you can see on this particular
slide, we've developed a lot of big

partnerships over the years, and we've
become really well recognized for

adhering to best practices and making
sure that it's science first with a,

with a heavy amount of attention on
utility and utilizability by end users.

So the company actually came into
being after both my partner and CEO,

Eric Krohner and I had spent a while
working as a value added reseller.

We used to package and market
other vendors' assessments.

And we recognize that there were some,
some gaps, some features missing and

a lot of limitations on the usability
of any of these other platforms.

And so about a decade or so ago,
we got together and thought, well,

what if we were to re imagine this?

How could we create an assessment
that was more engaging for

candidates and also easier to use
for recruiters, hiring managers, etc?

And that was really kind of
the origin of Logi-Serve.

Mike Callen: It's very interesting.

Hey, Chris, you, you
mentioned traits, not states.

And I, and I really like that.

That's an issue that I think a lot of us
who are not IO psychologists wonder about.

You know, how, how do we know when
something is a trait that is mostly

inherent, but can somehow be developed
versus some sort of aspect of behavior

that's just happening in the moment
or inspired by the situation, but

not necessarily something that's
a part of a person's personality?

Chris Cunningham, PhD: Yeah, that's
a, that's a great question, and it is

a, it is a difficult one to parse out
because, you know, that distinction

between trait and state is, is very
popular amongst researchers and

personality and individual differences,
but in practice, it can be quite

difficult to know or see the difference.

One way to look at it is that an
individual difference that's more of

a state or a kind of context specific
difference is one that only shows up in

reaction or response to specific stimuli
or specific set of contextual factors.

And as a result, that characteristic
might disappear or show up in a completely

different way if a different set of
stimuli or different context is provided.

What you see then in an actual practice
setting is that that aspect of a

person's behavior might not be real
consistent across different situations

or contexts because it is a response or a
reaction to aspects of that environment.

In contrast, a trait, something that's
a little more stable, is generally the

type of individual difference that is
a little bit more universal, a little

bit more consistent, generalizable.

It's the kind of aspects of
a person that you tend to see

regularly across different contexts.

And so usually when we think
about who a person is, or like,

what do they bring to a job?

Especially we're really quite interested
in the aspects of their personality

and their individual differences that
are more consistent because that's,

that's more of a reflection of who that
person is and what they're capable of.

Mike Callen: That's very interesting.

I really appreciate you sharing that.

It's not something we kind of had
on our bulleted list to discuss.

But it's, it is definitely
pertinent in terms of, you

know, the recruitment situation.

It also occurred to me as you were
describing it, that is it is very,

very much to me, at least a similarity
between the hard skills testing that

we do and the behavioral or soft skills
testing that you folks are targeting.

We, we would love to wrap our minds
around, you know, those things that

are traits that, that we can depend
upon the results of our testing.

And so it's really great that as you
sort of kick off this discussion that

you're doing so in a way that lets
us know that you're targeting these

performance dimensions or aspects of
personality that lead towards specific

performance on the job that are
more, more concrete, less temporary.

Chris Cunningham, PhD: So, yeah, yeah,
I would say that that's a, that's a

good way of kind of summarizing it.

And I would also just add, you know,
this, this idea of individual differences

ranging from completely fluid or
situational specific to completely

static and fixed there's, there's
quite a lot of variability in there.

And so that's actually one of the
reasons why we got so interested in

behaviorally oriented competencies in our
own assessment development, because we,

we really wanted to help organizations
identify I guess the simple way to put

it is individual differences that matter.

So we were less interested in abstract
trait labels that are just maybe, you

know, general descriptors of a person
and more interested in aspects that could

be closely linked to a motivation or
a likelihood of engaging in a specific

performance related behavior, and
especially when we target a domain like

service or interacting with others on
a regular basis, there's an opportunity

to think really concretely, as you said,
about what does good service look like?

Like, what does it take to demonstrate
service on a day in and day out

basis in a variety of roles?

If we can fixate and focus on those
characteristics, what's what's

really interesting is we can assess a
person's underlying sort of propensity

or likelihood of engaging in that.

We can also give hiring managers and
other talent leaders a chance to see

sort of the bench strength of their
team and to understand, you know, where

might we want to invest in a little
training and development to help shore

up some of these different behavioral
tendencies and skills that are needed.

So I think maybe an even simpler way
of thinking about how these two general

products support each other is if
you've got this underlying propensity

or competency, it really helps explain
that motivation, that intention,

that attitude toward demonstrating
these these different behaviors.

But to do that effectively, you
also might need skills to be able

to demonstrate these underlying
competencies in a way that comes

across as aligned with the specific
performance requirements in the job.

So this is sort of like a one two punch
might be a simple way of thinking about.

Mike Callen: Great.

Yeah, that's, that's very, very
interesting and very helpful.

Jenny Arnez: Yeah, I had a chance to
actually take a Logi-Serve test and

actually, before I ask this question,
why don't you explain a little bit

about what the Logi-Serve tests are?

Chris Cunningham, PhD:
Yeah, no, absolutely.

So in a Logi-Serve assessment, and I'll,
I'll I'll flip to a another graphic here

so you can get a sense of how this looks.

In a Logi-Serve assessment, we target
a number of competencies that we've

identified through careful study
and review of the evidence base,

the literature that's out there.

In the domain of service, there's close
to 60 or 70 plus years of really well

done research looking at what is it
that explains an individual's ability

or propensity to serve others well.

And we've consolidated that and
boiled that down to nine core

behavioral competencies that you
see listed around the outside gear

in this particular graphic here.

These competencies include things like
communication, adaptive problem solving,

interpersonal skills, motivation to
serve, self efficacy, so on and so forth.

But what makes the assessment
unique over and above that focus

on these generalizable competencies
is the way in which we assess them.

So unlike maybe a more traditional form
of assessment where you might see, you

know, a lot of items all in a, in a
set and you have to rate each one on

a scale of maybe strongly disagree to
strongly agree or something like that.

We have broken the assessment into
three different sort of activities,

and the first one candidates are
asked to reflect on past experiences

that they've had in life and in work
settings that would have allowed them

to build competence, build proficiency
and mastery in these different domains.

And then we ask them some questions about
different attributes of their personality

that they tend to display or show on a
regular basis that are ways that these

different competencies might surface or
manifest to others on a day to day basis.

And then the third protocol or
component to the assessment is

a series of situational judgment
or scenario type questions.

And these are designed in a way that
partners the sort of written scenario, the

text, with digital art that illustrates
and represents the actual work environment

that a assessment is being used for.

And with this particular last piece of
the assessment, individuals are now not

just telling us like who they are and
what they've done, but they're actually

showing what would you likely do.

It's a way of getting into the
intentions and the actual capacity

somebody has to make a proper behavioral
choice in the actual work setting.

The way the assessment ends up wrapping
up is that the data gathered across

all of these different prongs is then
incorporated, integrated together

to generate scores that reflect a
pretty deep understanding of the

relative strengths and weaknesses of
each candidate with respect to all

nine of those targeted competencies.

Mike Callen: That's it's very interesting.

I think one of the things that I
noticed in taking the Logi-Serve

assessment as well is that when you're,
you're dealing with the it's an,

it's an avatar based product, right?

You have sort of these illustrated
scenarios so that one can get

an idea of what the situation
is that they're operating in.

Oh, good.

You've got some.

Fantastic.

So in this situational judgment phase
the test takers presented with a scenario

and they're asked, you know, how would
you respond in this particular situation,

which I think is pretty typical.

One of the things that I really like
about your product, and I sort of suspect

that this might be a big part of the
secret sauce is that there's a follow up

that says how is the caller or customer
or, you know, the, the, the operative

person that you're dealing with in the
particular scenario going to respond?

Or how are they going to understand
your response or take your response.

I'm not probably not using the
right word, but it seems, you know,

one is directly what's the best
way to operate in this scenario.

But then when you put this other aspect
of it, it seems to be really more

of an empathetic type of situation.

You have to respond the way that
you're supposed to respond, but how

is it going to make that person feel?

Am I right about that?

Is that, is that a big
part of your secret sauce.

Chris Cunningham, PhD: Well, you're
right in recollecting that that's

a big component of these scenarios.

And, you know, it's interesting because
all of these prongs that I just reviewed a

minute ago, they're all so deeply entwined
that it's kind of difficult to tease

apart what the real secret ingredient is.

But I will tell you that it's
definitely a big component of it.

The reason is, as you've described,
Mike, there's there's kind of like a

two stage process to our scenarios.

Okay.

So with this type of scenario based
testing, it's really common to see

items where you're presented with
a scenario and asked, what should

you do or what's the best response
or what's the worst response?

That's not the way we approach it.

Instead, we ask, we give individuals
a scenario and then we give them four

choices and that the behavioral choices
or the options have all been graded in

terms of appropriateness that situation.

So it's kind of the good,
bad and the ugly responses.

And we look for the likelihood that each
candidate would respond to that situation

in each of the ways that are outlined.

So right off the bat with our item
sets, you're not just learning what

somebody thinks the right answer is,
you're actually getting a more complete

profile of what the person would
would likely do and would not likely.

Okay, so that's that's stage one.

We get a more complete picture of
what we would expect to see out of

that candidate in that scenario.

And then the second stage is let's
see if this person is able to think

through consequence of their action.

So here we get a chance to kind of
explore or peek under the hood of

their decision making capabilities.

And in a service provision context, that's
so critical because so much of providing

good service is about maintaining a
relationship with the other individual.

Even if it's very short term, there's
this development there of a connection.

And if if an individual is not able
to kind of think through, okay,

well, this is what I said I would
most likely do of those options.

Now, what do I think the consequence or
the reaction of the customer is going to

be or the patient or the client or the
guest is going to be on the other end that

allows us to see a little bit further.

So we refer to this as like
service forethought or the

ability to sort of anticipate.

Where does the story go from here?

And so at times we've also referred
to our scenario portions as like a

storyboarding activity, because in
essence, that's what it feels like to the

candidate is there's a little snippet.

Mhm.

Of a story unfolding here, and they're
getting a chance to jump into that

role for a minute, kind of explore it.

And then they move into a different story
that targets a different competency.

Mike Callen: Yeah, that that
is really very much based upon

the response that they choose.

However, they're reacting
to the situation.

So it, it to me seems like it shows
a degree of competence, but then if

you also are able to understand how
someone's going to respond, it, it

probably tends to suggest that they're
going to be better at continuing

that storyboard throughout time.

Yeah, yeah, very, very, very cool.

Chris Cunningham, PhD: And I'll also
note that this, this approach to

testing, I got to remind everybody that
we designed the whole test also be the

kind of an assessment that a candidate
would actually finish because one of the

other problems that plagues assessments
right now is that many individuals will

start, but not everybody will finish.

With our assessments, we generally
see completion rates above 90 percent

and some of our clients is 98%, which
is really high in this industry.

And part of the reason it's so high is
that everything is structured in a way

that it kind of grabs the attention
of the candidate, pulls them in, and

they want to see what happens next.

And before they know it, they're done.

Because the whole assessment, I
mean, it only takes most people

on average less than 20 minutes.

Usually it's around 15 to 20 minutes,
and that's that's a really kind of

focused assessment that yeah, it requires
more than two seconds, but you're also

getting a lot of information out of it.

And the candidates feel it like they
can see the relevance of the questions.

It's not abstract.

It's not way out there
in the ether somewhere.

They can, they can understand why
you want to know this about them

before you make a hiring decision.

Mike Callen: Yeah, absolutely.

And that, that face validity is so
satisfying for for, for, you know,

job seekers versus, you know, the
traditional personality tests.

I think the classic isn't it at the
combine they ask whether the players

prefer hot dogs to hamburgers or some,
some, you know, silly things like that.

Do you like to be in
crowded movie theaters?

Or do you like to go to, you
know, parties where there's

lots of people and loud music?

You know, what does that
have to do with this world?

And there might be some strong
corollaries to behavior, but it

doesn't feel really satisfying.

Whereas this feels incredibly satisfying.

And I think that's such
an important aspect.

Chris Cunningham, PhD: So, you know,
and you bring that up, Mike, and, and

one of the ways we explain this to a lot
of our partners and clients in the past

is, is using a graphic like this, and
we try to help people understand that

when you talk about the, the validity
of assessment, what you're actually

talking about is the ability of that
assessment to inform your ultimate

decisions so that your decisions are good.

So this is what's meant when
people say validity is a, is a,

is a property of inference, right?

It's not actually something
that's baked into the test.

It's what happens after the test.

Well, for that to work, you know,
not only do the candidates have to

appreciate why they're being asked
what they're being asked, because

then they're likely to respond more
honestly and truthfully and completely.

But your hiring managers and recruiters
also have to believe that the test

is measuring something relevant.

Otherwise, they're not going to use it.

You know, and then you also want to
make sure that the test is indeed

measuring what you say it is.

And that's where that cold
construct validation comes in.

And that piece is completely
absent from a lot of the other

assessments on the market.

And then, of course, you need to
be able to show that scores on the

assessment do predict or explain
variability in performance outcomes.

And that's that criterion part.

So what we've done over the years
is we've built a glass totally full

approach here to this challenge.

And we know that that's not.

Necessarily required in all
contexts, and we also know

it's not necessarily the norm.

But again, one of our goals is to try
to help people appreciate the value and

the importance of assessment science.

And that's, that's really been one of
our guiding principles from day one.

Mike Callen: It's very impressive.

We have some experience in the past
with an office and interpersonal

competence test that we created and
we decided to do, you know, very high

fidelity video situational judgment
where we filmed the scenarios and such.

And it was it's, it's been shocking
to realize what the shelf life is on

something like that, you know, you can
spend an awful lot of money creating

something like that only to find
that, you know, the hairstyles and the

fashions change so quickly that it's
maybe the juice isn't necessarily worth

the squeeze to go very high fidelity.

And I think that I would admit to
you that I've tended to be a little

snobbish about the avatar style testing
because to me it always seemed a little

cartoonish in comparison to strict
video situational judgment but it's not.

It's it's not that way at all.

And moreover, I think that
it's abundantly more updatable.

And in fact, you have instances
where you're able to as, as you can

see, in some of these examples here,
you're actually able to customize

certain elements within the scenario
for, you know, marquee clients.

I'm sure that's an upgrade.

But.

But that's really some spectacular
examples of, you know, what,

what it is that you can do here.

So, yeah, thank you for toggling through,

Chris Cunningham, PhD:
yeah, I just want to share.

I mean, and one of the things I want
to share with you is we, we took years

actually figuring out whether we were
going to go video, whether we were going

to use digital art, whether we were not
going to do any illustration at all.

And in addition to the cost
issue that you outlined, Mike,

there's also a speed issue.

You can't update video as fast
as you can update digital art.

There's a major cost issue.

There's also a delivery issue.

So like this assessment platform is meant
to be optimized for mobile delivery,

meaning over a mobile device, which, you
know, in some parts of the world, there's

still not widespread broadband access.

So delivering a video can really slow
things down and create a barrier.

But with these images, you know,
they can pop up and they look

consistently good on any kind of device.

I'll even share one quick
kind of funny story with you.

One of our very first clients, when
we configured the scenarios for them,

they were a larger retail operation.

We got everything set.

They loved it.

And then about a month later,
they did an org change.

And one of the biggest things
that happened is the color of the

uniforms changed from blue to red.

And they're like, we can't have
blue shirts in these images.

Like, Oh my gosh, I think we're
going to have to stop using the

assessment, but we really love it.

We're like, well, wait a minute.

Like, you just need the shirts
to be a different color.

And they're like, yeah,
that's, that's what we need.

And we're like, Okay.

Give us a half an hour.

Like, and that was it was done.

And so it's, it's not hard, you know?

In fact, that's why we built it this
way was to be able to work with,

create partnership with our clients.

Mike Callen: Give us a half an hour
at a hundred thousand dollars and

we'll take care of that for you.

Jenny Arnez: I have to say, I think
this style, this, the, the art

made it easier for me as a test
taker to enter into the story.

I had.

I hadn't taken a Logi-Serve test before.

And so I'm, I'm in there and I
thought, Oh, this is kind of exciting.

I was literally entering in.

Now, if I was watching a video,
I would have felt like an

outsider, just kind of observing.

Definitely.

It felt like I was becoming
part of some adventure.

It was a, it was a remarkable,
positive experience.

Chris Cunningham, PhD: That was actually
one of the inspirations for it was the old

choose your own ending storybooks, right?

Yeah, but the other thing we're doing
here and you know, it's again We're not

trying to trick people but the other
thing you should know about this format

is when we are conveying images like
this or like this or these we're able

to actually not just tell people this
is the kind of environment you'd be

in loosely we're able to render these
exactly like the work environment.

So this particular illustration here this
comes exactly from a client call center.

Like we, we took, this is a high
customization, obviously, but we got, we

went in, we took photos, we rendered it.

So the posters actually
reflect cultural cues.

The technology on the desk is the actual
technology that the person's going to use.

The color scheme is the color scheme.

And so.

And so what's happening is you're
also getting a chance to queue and

send signals to the individual, which
helps improve the relevance of what

they're about to say response wise.

So it's good for the candidate, but
it's ultimately good for the client

because you're getting a higher quality
piece of data coming out the other end.

Jenny Arnez: That's fascinating.

Mike Callen: The video situational
judgment tests that that we did you

know, a big part of the scripted aspect
of it was getting the right camera

angles on the operative person and being
able to express these nonverbal cues

that you want to express, you know,
distaste, displeasure, anger, whatever.

And so that was always something that
we perceived as a huge advantage.

But it's interesting when you're
clicking through the examples that you

have there of some of the different
characters and scenarios, it's very

easy to see these nonverbal cues,
you know, to the same extent that you

would be able to see them in video.

And ironically, I, as I'm thinking
about them, I realized, you know, that

we really, you know, paused on that
person, almost in the same way that that

your static screens are in order to be
able to be sure that we were properly

conveying that that feeling or sense.

And, and so at any rate, again, that's
just, you know, my way of, of really,

you know, complimenting the process
that you folks have invested so much

into because it's very, very effective.

Chris Cunningham, PhD:
Yeah, I appreciate that.

And I know our team does too.

And I will tell you on that last
comment you made, it's interesting.

We really struggled with that for a while
until we realized, listen, you know,

it might be neat to capture the nuance
of nonverbal expression in a video.

But when that gets reduced to the
size of an image on a smartphone,

you're not going to see it.

And so digital art allows us to, and
you might notice like some of our

graphics are, they seem a little like
exaggerated when they're blown up to full

screen, but when they come down to the
level of a graphic on your smartphone,

it looks natural, it looks normal.

And that's, and that was part
of the control that we could get

with this digital art approach.

So yeah, a lot of thought went into it.

I can't, I can't say enough for
our digital art team and their

ability to capture the essence.

Like we would draft items along
with descriptions of here's the

emotions that need to be conveyed.

This is how these emotions are
generally conveyed in a universal way.

Let's capture this in the images.

And we went from there.

Sponsor Break: And we'll be right
back after a word from our sponsor.

Ready to revolutionize your HR strategies?

Head over to testgenius.com to discover
our latest tools and solutions designed

to streamline your hiring processes.

Jenny Arnez: Can you talk a little bit
about how an HR manager or director

might use These assessments in both
pre-hire and then also you've mentioned

post-hire and employee development.

Chris Cunningham, PhD: Sure.

Yeah, sure.

I'm happy to do that.

So I'll start here.

You know, by focusing on competencies,
what we're doing is we're digging

into sort of the foundational
attributes or aspects of a person

that enable them to actually
demonstrate competence on the job.

So in other words, if I if I have these
underlying competencies that I'm more

likely to actually do the right thing
behaviorally in the work environment

and in the service context, the way
this might look in practice is something

like what's represented on the screen
right now, where, if I've got employees

that are competent under an underlying
way, and they're doing the right thing

behaviorally, then that positively
impacts the customer, the guest,

the client, the patient experience.

And that's really good for business.

Okay.

And then the other thing is
when my employees are doing the

right thing behaviorally, it
also improves the operational

efficiency of that organization.

And that has an impact on the bottom line.

So a simple value chain argument like
this helps to illustrate for many

decision makers and stakeholders in an
organization why we want to understand the

underlying competence of our workforce.

From this, we can then expand that
conversation and say, well, listen,

you know you can't probably just hire
top competency people because your,

your labor pool, your, your applicant
flow probably isn't strong enough to

allow you only to take top scores.

So you're going to have more
variability within your team.

Wouldn't it be neat if you could
figure out where they're stronger,

where they're weaker, and then direct
some development initiatives at those

individuals to see if we could strengthen
their overall level of confidence.

And so for post hire applications, that's
often one of the ways this assessment is

used, is we'll generate a report, you get
to see the profile of your individuals,

but you can also aggregate and see group
level sort of summary statistics and

breakdowns of scores, and then you can
work from there to develop training plans.

Here's an example of some of the reporting
that comes out of a typical assessment.

And you can see we've also put
a lot of time and effort into

making this really simple.

So this is not one of those
assessments where, you know, your

users have to go through six months
of training and get certified before

they can do anything with the test.

Like we've actually done studies where
we've gone to HR conferences and we've set

up a little computer lab and we brought
in HR managers and said, Hey, you know

here's here's access to the platform.

Now make a good decision and
justify how you made that

decision in about five minutes.

They can make a good decision because
people understand that gold is better

than silver is better than bronze.

Okay.

But they also understand that from a
medalist perspective, even a gold medalist

has to keep working to stay strong and
a silver medalist has room to grow, but

they're still potentially really valuable.

And a bronze medalist also there might
be worth seriously looking at, especially

if they've got some competencies
that are in a stronger territory.

So we use this type of reporting
methodology also to keep the focus from

day one, not on screen in screen out that
kind of dichotomous decision, but we use

this kind of reporting to always get our
talent managers thinking, okay, this is

what I'm about to potentially employ.

And I want to be able to work with
this individual from day one until

the day they leave the company to
help them reach their full potential.

Why?

Because if I invest in my talent, they
create value for the organization.

So that was always part of the plan,
you know, from the beginning and all

the reporting is designed this way
to kind of facilitate that thinking.

That's also why sometimes this
test is very popular to be used to

identify potential for promotion
because you're looking for people

that are showing maybe growth or
a potential area for achievement.

And then you can put them on a track for
development or some rotational program.

Sometimes it's helpful in that way too.

Mike Callen: Yeah, I, this
is really important aspect.

When we started talking with Eric many,
many years ago about working with you

folks, you know, this is one of the
agreement points that we wanted to make

sure that we shared, which is that when
we're looking at hard skills aspects,

we're looking at skills, abilities,
personal characteristics that are

required day one for success on the job.

So it's very easy to have that
dichotomous situation that you described

where you can clearly have the skills
and abilities necessary for success.

Or maybe you clearly don't.

And so it's really easy to make these
decisions when we're looking at these soft

skills aspects, I love the bronze, silver,
silver because everybody's a winner.

The question is, is what's the
attributes that you do possess at

this point in time that will dovetail
nicely with your bevy of hard skills

that you've already shown to possess.

So I really like the way that these,
you know, two really different aspects

work together towards the same result,
which is recruiting people who are

going to "A" - be successful in the
job and "B" - hopefully continue to be

successful there for many years to come.

Chris Cunningham, PhD: Right, right.

And I'll show you to like
in a more general sense.

This is how that scoring context
gets broken out in most reports.

And so you'll see that our middle
range, our norm, if you match the

benchmark, so to speak, for a given
assessment, then you're going to land

squarely in the silver territory.

But if you see a gold medalist or a
high silver person, somebody has a score

above 500, that's likely a very strong
candidate with respect to that competency.

And so the, the, the gist or the guidance
for a user is quite simple, like aim high.

Okay, but don't feel like you have to
wait for all of your applicants to test

so you can rank order them at the end
of the process because we know that's

not feasible for most organizations.

So instead, you know, look for people that
land in that silver and gold territory

and give them serious consideration.

And if your applicant pool is not strong
enough, and all you end up with is a

lot of bronzes, but they might have some
strengths in the silver gold area take

a close look at them and ask yourself,
are we prepared to work on building

some of these bronze competencies?

Because if we are, then it might
be a really great candidate.

Mike Callen: Yeah.

And again, especially if you have
other, if they possess other hard

skills, at a very high level.

And you know, there are certain tests
that help to measure the trainability

aspect, you know, your comprehension
and those kinds of aspects.

And so if you have somebody, you know,
who like in your example is a high

bronze, has good skills and abilities
appears to be trainable, but would be

a great addition to your team then it
makes it a lot easier to take a chance

or, or maybe dip the ladle a little
bit lower into the different tiers

that you have among your candidates.

Chris Cunningham, PhD: Absolutely.

Absolutely.

And, and the one thing I just wanted
to add with this particular graphic,

I'll just throw it up on screen here,
is you'll you'll notice and this

actually comes from an actual client.

And so this is a representation of
how an assess how this assessment

can differentiate between strong
performers and everybody else.

And you'll notice that it's not
always the case that all nine

competencies are equally influential
in determining who a strong player is.

So that's another piece that has to
be taken into account for any given

client any given users Like maybe they
did have a bronze in this example here.

Maybe they had a bronze in proactivity,
which is down the lower left corner.

Well, you know what?

That was not a strong
differentiation point.

So I wouldn't lose sleep over that.

On the other hand, if, if we have an
individual with a bronze and motivation

and interest, which is just to the
right of proactivity, you notice

the gap between the top performers
and everybody else is massive there.

I would be really worried at that point.

If, if they were bronze medalists,
I only want silver and gold when it

comes to motivation and interest.

So my point is people using the assessment
have a pretty good understanding in

their organization of what what is
critical, what is not so critical.

And the beauty of this assessment is
it gives you that ability to make that

more nuanced decision because it isn't
pass fail like you're saying, like, but

it does give you guide rails, right?

And it makes it pretty easier,
easy for you to kind of figure out.

Okay, what is the balance trade off risk?

We're willing to take with this candidate.

Jenny Arnez: So are you recommending
that HR people, managers, they look at

these nine competencies and they, they
rank them as to what's most important

to them and for a given position?

Chris Cunningham, PhD: So, sometimes
that's helpful approach but I'll

show you again in the main report.

The, the number that most
people use is the overall.

That's the one that's at
the top of the report.

And what that is, is an
aggregation of all nine.

And the, it's, it's beyond our
scope today to talk about exactly

how these scores are generated.

But I will say to you that these
scores are, there are individual

scores for every competency.

And there are composites that combine
them in certain weighted fashions,

and then there's an overall composite.

And by and large, most of our users
first look at the overall score to

do some initial filtering, and then
they go down to the nuanced breakout.

And you can even see in this mock up
here on the screen, this particular

candidate is not a strong player.

But they happen to be okay, a
little okay with proactivity.

So I could look at this and say, well, you
know, I, I would consider this a higher

risk person, but if they, if they have
productivity, then that's a really great

attribute, especially if I'm willing and
able to invest a little bit in helping

them see gaps that they need to address.

And so I don't know that I
personally would take a risk on

this particular candidate, but if I
got into a bad situation, I might.

Mike Callen: Yeah, you, it gives
you the awareness of the risk.

So if you're in a situation where,
you know, you've it's a little crude,

but a lot of the people that we work
with talk about butts in the seats.

You got to get butts in the seats.

And you know, sometimes you need to
have a human being in there and it

may not be your first or second or
third choice, but you've got to get

somebody in there doing the work.

And so you have a good idea of what
type of efforts going to be necessary

in order to, to bring them along.

You know, to that end, Jenny
asked you about post hire.

And I know that for a lot of positions
there are preceptor or trainee.

Or sorry, trainer to trainee
relationships, and there you go.

That's one of your aspects here.

Do you find, or do you recommend that
having the preceptor or trainer go through

take the test and then trying to, you
know, match behavioral types between

those would make the training slash
onboarding process more successful because

you're matching like people together?

Or is there any research
that you've done on that?

Chris Cunningham, PhD: I am following
what you're outlining, and I think

there probably is some evidence that
that approach might help, especially

if the, the training and development
model is, is kind of a mentoring

forum where you're trying to match
people with you know, with, with

that sort of mindset and approach.

I can't tell you that we've done
research on that particular model

yet, because most of our clients.

Have wanted to use this to
help make better decisions

about new talent or talent.

They want to develop.

There's some that have started using
the assessment to guide promotion

decisions, but we haven't seen
a ton of interest in using it to

pair partner employees together.

So I think it's, I think it's a
great illustration of another way

in which this competency approach is
so flexible and is really designed

to support good talent management.

It's not simply a pre hire
selection tool like that.

That's one point I think we just
need to emphasize is that there's

a lot of opportunity here to
use this both pre and post hire.

And that was by design.

So we'd really love to see clients
use it for all of its richness.

Mike Callen: Excellent.

Well, you know, if that comes up down
the road, then you know, maybe that'll

be a great opportunity to have you back
and talk a little bit more about that.

I know that, like in nursing, for
instance, that preceptor trainee

relationship is super important.

And so you know, because people do
learn differently, there could be

a benefit and, you know, sort of
matching them up style to style.

But at any rate, it was just something
that was that I was curious about.

Jenny, do you have anything
that you wanted to bring up now?

Jenny Arnez: I I just a
general, just a comment.

Taking the assessment, the whole
approach was the most positive

assessment I've ever taken.

It typically, you know, when you go to
take a test or an assessment in, say,

Excel, it's a little nerve wracking.

You can, you can get a little bit
stressed, but I didn't feel that in this.

And so kudos to to you all for
developing something like this.

It's amazing.

Chris Cunningham, PhD: I appreciate that.

And I don't have a graphic to add here,
but I, I will share with you because

we've actually done some direct testing
of candidate experience and reactions and

a lot of times what we hear is that more
than 90 percent of our applicants describe

the assessment as fun, and 90 percent or
more actually describe it as easy, and

people have tried to tell me at times that
an assessment shouldn't be easy and then

I say to them, how's that working for you?

Because the thing is, like, if
our goal is to trip and trick and,

you know, make people feel stupid.

Then as an industry,
we've done pretty great.

But if our goal is to learn about
people and make better decisions so

that fit is ultimately accurate, then
we need an assessment that people

are going to be willing to take and
actually give their best selves too.

So, so I think that that's
really really helpful.

The other interesting metric I would just
share with you all is we've also tracked

sort of a net promoter effect with this
assessment and in some of the engagements

we've had with clients, we've asked
people, "Hey, you know, now that you've

had this assessment experience how likely
would you be to recommend a friend of

yours to apply for a job here as well?"

And again, more than 90 percent of users
would recommend that company because

of their assessment related experience.

And I don't think we spent enough time
talking about that in this industry

because you gotta understand like if
we create a front end experience that's

abysmal we're also damaging our ability
to recruit talent because thanks to

the power of the internet, right?

Like if one person has a bad experience,
the rest of the world knows about it.

So having a good front end experience like
this, having a good internal experience

like this, I think it really matters
now more than it probably ever had.

Mike Callen: We do at least 50 percent
of our business in the 911 emergency

services, police, fire dispatch space.

It's a huge market for us.

And in this post COVID world it's
been uncanny how every conference

that we go to, there are big sessions
that are led by roundtable groups

of people who are talking about
how do we bring more people in?

How can we improve our situation
so that we get, you know, a little

bit of momentum going in our,
communities, you know, towards, you

know, bringing more people into here?

It's, it's become a more and
more challenging situation.

And you know, I, I do tend to kind of
come, you know, from the side where at

least on the hard skills we've got to be
able to ask them some tough questions.

And, and to a certain extent, you know,
if you can't like to, to use Jenny's

example, if you can't work Excel to a
very high degree and the job requires

you to work Excel to a very high degree,
then, you know, you shouldn't feel

as though you, you passed the test
with flying colors if you failed it.

But when you're dealing in the, this
behavioral soft skills realm, you know,

that's really kind of an opportunity.

People are what they are, you know, to
a great extent at this point in time.

And so to have an opportunity to really
sort of feature yourself and, and do

so in a way that that is affirming and
positive is, is really is really great.

And, and actually what it sort of leads
me to think might be a good best practice

when we're putting tests together within
within our platform, including your

tests, you know, that we may want to, to
recommend to people that they go through

and they put the hard skills tests first
and let people finish with something

that's a little more fun and is going
to leave them with maybe a little bit

better memory of the testing process.

Chris Cunningham, PhD: So yeah,
yeah, that's something that would be

interesting to explore because you know,
the other the other approach is to warm

them up with something easy, right?

And they get them excited
about well, okay, they've

asked me good questions so far.

So let's let's go a little further.

But, you know, what I think is
important here to to point out

is, you know, worrying about the
look, the feel, the experience, the

assessment, this is not like meaningless
conversation for an academic audience.

Like, we got to also remember that
unfortunately, assessments have also

had a pretty bad history of maybe
discriminating against certain groups

and not really leveling the playing
field as much as they're supposed to.

And that's something else that
we've spent a lot of time working

with in the development of our
tool especially is making sure that

it does level the playing field.

And we've actually seen some really strong
evidence of this in a number of studies.

And what I mean by that is that this test
does not show preference to any group.

So if you're a member of a minority or
protected group, you have just as much

chance of scoring well on this as somebody
who's a member of a majority group.

It's not a cognitive test, first of all,
okay, and it's focused on competencies

that you could have developed in a variety
of different life domains and experiences.

Why I'm emphasizing this is that we really
do, as an industry, need to do more to

think about how we package together these
screening experiences and really just

keep asking ourselves, like, are we asking
questions that that get at attributes

that need to be present day one, week
one, or are we focusing on things

that we can actually build over time?

And I know this is a, is a hard challenge,
especially when you're working with

skills testing, but I know you guys
have done a lot to tease that apart

with your clients and make sure they
understand that when they're putting

these assessments in place as well.

Mike Callen: Definitely.

We have we haven't talked much about
the structure of your assessments in

terms of the library itself, but you
have a sort of a service focus, and then

there's a sales focus, and then there's
a managerial leadership potential focus.

And then you have many different
industries or verticals that

are represented with each
of those particular areas.

And then you can drill down further
and you can choose specific job titles

that are very standard across the
country, or maybe even around the world.

All told I think that we have about
shoot 220 or so of the Logi-Serve

tests available in our library
to our clients who have that

particular add on service activated.

So the, the new almost every new
client that comes on board since

the integration has been done will
be getting this particular service.

And then those clients who signed on
prior to it have the opportunity to

upgrade their subscription to include it.

But it's a very, very robust
library that you've you've made

available and very impressive.

Did I leave out any of the
details about the library or

anything that you'd like to add?

Chris Cunningham, PhD: No, I think
that's a good summary of an explanation

of the value add that this brings.

And you all already have a wonderful
product that can really help

companies make better decisions.

And we're really happy and excited to
become a partner that can help you take

it to another level with your clients.

We've we've seen a lot of success
using this product as a standalone.

So we know that when it's partnered
with another well designed product,

the impact will be even greater.

And we're just excited to kind of
see some of these impacts happen.

Mike Callen: Excellent.

As are we for sure.

One of the things, areas that

I reflect upon is the,
the leadership side of it.

And I don't know if you have any
particular insights on this, but

having grown up in, in sales very
often organizations will, you

know, promote their best performing
salespeople to sales managers.

And sometimes that can have
good results and sometimes

it can kind of be disastrous.

Sometimes the best salespeople
aren't the best sales leaders.

And sometimes the best sales leaders
aren't the best sales people either.

So being able to add something to
the conversation that gives you some

insight and, and intelligence into,
you know, deciding who it is that

you promote into these situations
can be, can be very, very helpful.

And some of the smaller organizations
of which you know, most of the

businesses in America are you know,
making the wrong decision in this

regard, you know, especially for
instance, promoting a really good

sales person to sales management and
having them fail can be disastrous

to, to that particular organization.

Chris Cunningham, PhD:
Yeah, yeah, that's so true.

And, you know, to add a couple of
layers on this, I mean when you take

a very strong individual contributor
like the examples you've been giving

Mike and you put them into a managerial
role, even if they are effective as a

manager, you've lost the value of their
performance as an individual contributor.

So it's a real big gamble and we have
to be careful because sometimes the

characteristics and attributes that make
individuals powerful as an individual

can actually hinder them from being
good at working well to support others.

It's, it's a different competency
set to be a leader and a manager

than it is to be a sole contributor.

So that's exactly the rationale behind
why we built that, that vertical out.

And you know, in that particular,
assessment we've also seen it be used

a lot to help organizations increase
diversity and their leadership potential

and their leadership development programs.

Because the problem with the model
that you shared before, Mike, that

is so common, is when we get in the
habit of identifying our current top

performers or we ask managers to, you
know, identify talent, it tends to

happen through a similar to me process.

Like, you know, the manager
says, Oh, well, yeah, Joe reminds

me of myself 20 years ago.

I think He's the one to tap.

And of course, the problem
with that is that that

automatically creates homogeneity.

And it really damages an organization's
abilities to go in a different

direction or to really open that
door for more people to consider it.

So I mentioned leveling
the playing field before.

It, in leadership, we see that as a really
big need in many organizations because

they just don't have a process in place
that's scalable to figure out who do

we want to fast track for development?

Who do we want to put on that
leadership track so that we're

ready when those roles open up?

Mike Callen: Yes, very, very good points.

Well, I, Jenny, don't
have any other questions.

I'm looking through my, my list
of reminders that I had that

I wanted to ask Chris about.

So, Jenny, do you have any other
questions that you'd like to ask

or comments you'd like to make?

Jenny Arnez: No, I don't actually, I feel
like we're kind of we're winding down.

We're at a good stopping point.

I think for this conversation,
I mean, I definitely have other

thoughts that I'd like to ask you
sometime Chris about AI and we kind

of touched on this in a previous
conversation, but that's not for today.

That's a, that's a whole different, street
that we'd be driving down, but I, I just

am so grateful that you joined us today.

And I've thoroughly
enjoyed this conversation.

Any final thoughts from
either one of you guys?

Chris Cunningham, PhD: I'm just
appreciative we have the chance to

have these types of conversations.

A lot of times what holds other
clients back is lack of awareness

of some of the complexities and
nuances that operate under the hood.

And then just being able to talk about
some of these topics, it may not be

something that was on somebody's mind,
but then when they learn about it, they're

like, yeah, you know, that was on my mind.

I just didn't know what to call it.

So it's important to have these
types of discussions and to

raise these types of issues.

So thank you both for
putting this together.

And I'm always happy to, to
take that trip down those other

streets when you guys are ready,

Mike Callen: That's great.

Yeah.

I, I, I really appreciate you, you
being here today and you know, we,

we hope that this content, you know,
meets the needs of new HR people.

People are getting established and really
trying to learn, you know, about different

aspects of testing and selection.

But like I told Jenny before we
started today, I was really excited

about this because, you know,
it's fun to geek out a little bit.

And you know, when you have, you have
somebody that's a chief science officer,

I mean, it sounds like you, you started
in Star Trek and, you know, you've just

kind of graduated to where you are right
now, but it's, it's really great to to

be able to pick your brain and and hear
your opinion on a lot of these things.

As I mentioned, we've been
in conversations with Eric

for many, many years now.

When we first started talking about
integrating our products, you know, we

had an installed product and it wasn't
something we could do at that time.

But we've persisted in the conversation.

And so, you know, now we're here.

We've launched we've got the generic
off the shelf products available.

We're going to work together to create
a market killer product for, you know,

the 911 EMS police and fire space.

And we're super excited
about that as well.

And so anybody who's listening to this if
you're interested, I'm sure we'll have a

form or, or some sort of call to action
available at the end of the video here.

If you're an existing client and
you'd like to, you know, upgrade

and add this incredible line of
testing to your subscription.

You can reach out to us and
we'll be happy to get you set up.

We can literally get that launched
for you in about five or 10 minutes.

And if you're not a client and you'd
like to talk more about being one and

being able to take advantage of this
incredible Logi-Serve product along

with our hard skills tests we'd love
to chat with you about that as well.

Again, Chris, thank you so much for
being so generous with your time today.

We really appreciate you
you know, sharing with us.

So thank you very much for that.

Chris Cunningham, PhD: Likewise.

Jenny Arnez: And we just want to say
thank you to our viewer and our listener.

We're sure glad you joined us today and we
hope to connect in another place and time.

Thank you everybody.

Mike Callen: Thank you.

Podcast-Outro: Thanks for tuning in to
Testing Testing 123 brought to you by

TestGenius and Biddle Consulting Group.

Visit our website at testgenius.com
for more information.