Exploring Behavioral Testing with Logi-Serve
Podcast-Intro: Welcome to
Testing, Testing 1-2-3, a podcast
brought to you by TestGenius.
Jenny Arnez: Welcome everybody.
My name is Jenny Arnez.
I'm your host for today's podcast.
I have two gentlemen with me today.
I have Mike Callen.
He's our VP of products at TestGenius.
I'm also part of the
TestGenius team and we have Dr.
Chris Cunningham.
He's the chief science
officer from Logi-Serve.
Mike, you want to go ahead
and introduce yourself?
Tell us a few things.
Yeah.
Mike Callen: Hi, thank you, Jenny.
I'm, I'm Mike Callen, as you
said, and I'm the VP of products
at, at TestGenius, which is a
product of Biddle Consulting Group.
And TestGenius is a skill and ability
testing platform that handles a lot of
different Positions and we have been
mostly engaged in doing concrete, hard
skills testing over the years and through
an integration with Logi-Serve with
whom for whom Chris Cunningham works.
We now have the ability to add a
really substantial library of soft
skills tests or behavioral tests.
And so with that, we want to turn it
over to Chris to introduce himself.
Chris Cunningham, PhD: Yeah, thanks guys.
I appreciate that.
Yeah.
So I'm, I'm Chris Cunningham.
I'm the Chief Science
Officer at Logi-Serve.
And what that means is I use a background
in industrial and organizational
psychology to guide the development
of our assessments and the ongoing
validation and management of those
different testing libraries the way
that they're applied with our clients.
And as Mike alluded to we are
focused in more of a soft skill
space with these assessments.
We tend to focus on competencies,
which you can think of as
complex individual differences.
They're more than just traits,
but they're not states.
So what that means is they're fairly
stable within individuals, but they
can be developed with concerted effort,
focused effort, and these particular
competencies that we target tend to
focus on behaviors associated with a
person's underlying service orientation.
And that's one of the reasons why this
partnership is potentially so powerful.
Jenny Arnez: Wonderful.
Mike Callen: Absolutely.
Jenny Arnez: Chris, can you tell
us a little bit about Logi-Serve?
Chris Cunningham, PhD:
Yeah, I'd be happy to.
And so again, just for the visual
learners out there here's, here's a
bit of a representation of some of the
things we've been up to as a company.
We've been around for
a little over a decade.
We are a provider of talent management and
assessment products that we have developed
in house and are purpose built to help
organizations take a more competency
focused approach to their screening
and hiring and developing of talent.
As you can see on this particular
slide, we've developed a lot of big
partnerships over the years, and we've
become really well recognized for
adhering to best practices and making
sure that it's science first with a,
with a heavy amount of attention on
utility and utilizability by end users.
So the company actually came into
being after both my partner and CEO,
Eric Krohner and I had spent a while
working as a value added reseller.
We used to package and market
other vendors' assessments.
And we recognize that there were some,
some gaps, some features missing and
a lot of limitations on the usability
of any of these other platforms.
And so about a decade or so ago,
we got together and thought, well,
what if we were to re imagine this?
How could we create an assessment
that was more engaging for
candidates and also easier to use
for recruiters, hiring managers, etc?
And that was really kind of
the origin of Logi-Serve.
Mike Callen: It's very interesting.
Hey, Chris, you, you
mentioned traits, not states.
And I, and I really like that.
That's an issue that I think a lot of us
who are not IO psychologists wonder about.
You know, how, how do we know when
something is a trait that is mostly
inherent, but can somehow be developed
versus some sort of aspect of behavior
that's just happening in the moment
or inspired by the situation, but
not necessarily something that's
a part of a person's personality?
Chris Cunningham, PhD: Yeah, that's
a, that's a great question, and it is
a, it is a difficult one to parse out
because, you know, that distinction
between trait and state is, is very
popular amongst researchers and
personality and individual differences,
but in practice, it can be quite
difficult to know or see the difference.
One way to look at it is that an
individual difference that's more of
a state or a kind of context specific
difference is one that only shows up in
reaction or response to specific stimuli
or specific set of contextual factors.
And as a result, that characteristic
might disappear or show up in a completely
different way if a different set of
stimuli or different context is provided.
What you see then in an actual practice
setting is that that aspect of a
person's behavior might not be real
consistent across different situations
or contexts because it is a response or a
reaction to aspects of that environment.
In contrast, a trait, something that's
a little more stable, is generally the
type of individual difference that is
a little bit more universal, a little
bit more consistent, generalizable.
It's the kind of aspects of
a person that you tend to see
regularly across different contexts.
And so usually when we think
about who a person is, or like,
what do they bring to a job?
Especially we're really quite interested
in the aspects of their personality
and their individual differences that
are more consistent because that's,
that's more of a reflection of who that
person is and what they're capable of.
Mike Callen: That's very interesting.
I really appreciate you sharing that.
It's not something we kind of had
on our bulleted list to discuss.
But it's, it is definitely
pertinent in terms of, you
know, the recruitment situation.
It also occurred to me as you were
describing it, that is it is very,
very much to me, at least a similarity
between the hard skills testing that
we do and the behavioral or soft skills
testing that you folks are targeting.
We, we would love to wrap our minds
around, you know, those things that
are traits that, that we can depend
upon the results of our testing.
And so it's really great that as you
sort of kick off this discussion that
you're doing so in a way that lets
us know that you're targeting these
performance dimensions or aspects of
personality that lead towards specific
performance on the job that are
more, more concrete, less temporary.
Chris Cunningham, PhD: So, yeah, yeah,
I would say that that's a, that's a
good way of kind of summarizing it.
And I would also just add, you know,
this, this idea of individual differences
ranging from completely fluid or
situational specific to completely
static and fixed there's, there's
quite a lot of variability in there.
And so that's actually one of the
reasons why we got so interested in
behaviorally oriented competencies in our
own assessment development, because we,
we really wanted to help organizations
identify I guess the simple way to put
it is individual differences that matter.
So we were less interested in abstract
trait labels that are just maybe, you
know, general descriptors of a person
and more interested in aspects that could
be closely linked to a motivation or
a likelihood of engaging in a specific
performance related behavior, and
especially when we target a domain like
service or interacting with others on
a regular basis, there's an opportunity
to think really concretely, as you said,
about what does good service look like?
Like, what does it take to demonstrate
service on a day in and day out
basis in a variety of roles?
If we can fixate and focus on those
characteristics, what's what's
really interesting is we can assess a
person's underlying sort of propensity
or likelihood of engaging in that.
We can also give hiring managers and
other talent leaders a chance to see
sort of the bench strength of their
team and to understand, you know, where
might we want to invest in a little
training and development to help shore
up some of these different behavioral
tendencies and skills that are needed.
So I think maybe an even simpler way
of thinking about how these two general
products support each other is if
you've got this underlying propensity
or competency, it really helps explain
that motivation, that intention,
that attitude toward demonstrating
these these different behaviors.
But to do that effectively, you
also might need skills to be able
to demonstrate these underlying
competencies in a way that comes
across as aligned with the specific
performance requirements in the job.
So this is sort of like a one two punch
might be a simple way of thinking about.
Mike Callen: Great.
Yeah, that's, that's very, very
interesting and very helpful.
Jenny Arnez: Yeah, I had a chance to
actually take a Logi-Serve test and
actually, before I ask this question,
why don't you explain a little bit
about what the Logi-Serve tests are?
Chris Cunningham, PhD:
Yeah, no, absolutely.
So in a Logi-Serve assessment, and I'll,
I'll I'll flip to a another graphic here
so you can get a sense of how this looks.
In a Logi-Serve assessment, we target
a number of competencies that we've
identified through careful study
and review of the evidence base,
the literature that's out there.
In the domain of service, there's close
to 60 or 70 plus years of really well
done research looking at what is it
that explains an individual's ability
or propensity to serve others well.
And we've consolidated that and
boiled that down to nine core
behavioral competencies that you
see listed around the outside gear
in this particular graphic here.
These competencies include things like
communication, adaptive problem solving,
interpersonal skills, motivation to
serve, self efficacy, so on and so forth.
But what makes the assessment
unique over and above that focus
on these generalizable competencies
is the way in which we assess them.
So unlike maybe a more traditional form
of assessment where you might see, you
know, a lot of items all in a, in a
set and you have to rate each one on
a scale of maybe strongly disagree to
strongly agree or something like that.
We have broken the assessment into
three different sort of activities,
and the first one candidates are
asked to reflect on past experiences
that they've had in life and in work
settings that would have allowed them
to build competence, build proficiency
and mastery in these different domains.
And then we ask them some questions about
different attributes of their personality
that they tend to display or show on a
regular basis that are ways that these
different competencies might surface or
manifest to others on a day to day basis.
And then the third protocol or
component to the assessment is
a series of situational judgment
or scenario type questions.
And these are designed in a way that
partners the sort of written scenario, the
text, with digital art that illustrates
and represents the actual work environment
that a assessment is being used for.
And with this particular last piece of
the assessment, individuals are now not
just telling us like who they are and
what they've done, but they're actually
showing what would you likely do.
It's a way of getting into the
intentions and the actual capacity
somebody has to make a proper behavioral
choice in the actual work setting.
The way the assessment ends up wrapping
up is that the data gathered across
all of these different prongs is then
incorporated, integrated together
to generate scores that reflect a
pretty deep understanding of the
relative strengths and weaknesses of
each candidate with respect to all
nine of those targeted competencies.
Mike Callen: That's it's very interesting.
I think one of the things that I
noticed in taking the Logi-Serve
assessment as well is that when you're,
you're dealing with the it's an,
it's an avatar based product, right?
You have sort of these illustrated
scenarios so that one can get
an idea of what the situation
is that they're operating in.
Oh, good.
You've got some.
Fantastic.
So in this situational judgment phase
the test takers presented with a scenario
and they're asked, you know, how would
you respond in this particular situation,
which I think is pretty typical.
One of the things that I really like
about your product, and I sort of suspect
that this might be a big part of the
secret sauce is that there's a follow up
that says how is the caller or customer
or, you know, the, the, the operative
person that you're dealing with in the
particular scenario going to respond?
Or how are they going to understand
your response or take your response.
I'm not probably not using the
right word, but it seems, you know,
one is directly what's the best
way to operate in this scenario.
But then when you put this other aspect
of it, it seems to be really more
of an empathetic type of situation.
You have to respond the way that
you're supposed to respond, but how
is it going to make that person feel?
Am I right about that?
Is that, is that a big
part of your secret sauce.
Chris Cunningham, PhD: Well, you're
right in recollecting that that's
a big component of these scenarios.
And, you know, it's interesting because
all of these prongs that I just reviewed a
minute ago, they're all so deeply entwined
that it's kind of difficult to tease
apart what the real secret ingredient is.
But I will tell you that it's
definitely a big component of it.
The reason is, as you've described,
Mike, there's there's kind of like a
two stage process to our scenarios.
Okay.
So with this type of scenario based
testing, it's really common to see
items where you're presented with
a scenario and asked, what should
you do or what's the best response
or what's the worst response?
That's not the way we approach it.
Instead, we ask, we give individuals
a scenario and then we give them four
choices and that the behavioral choices
or the options have all been graded in
terms of appropriateness that situation.
So it's kind of the good,
bad and the ugly responses.
And we look for the likelihood that each
candidate would respond to that situation
in each of the ways that are outlined.
So right off the bat with our item
sets, you're not just learning what
somebody thinks the right answer is,
you're actually getting a more complete
profile of what the person would
would likely do and would not likely.
Okay, so that's that's stage one.
We get a more complete picture of
what we would expect to see out of
that candidate in that scenario.
And then the second stage is let's
see if this person is able to think
through consequence of their action.
So here we get a chance to kind of
explore or peek under the hood of
their decision making capabilities.
And in a service provision context, that's
so critical because so much of providing
good service is about maintaining a
relationship with the other individual.
Even if it's very short term, there's
this development there of a connection.
And if if an individual is not able
to kind of think through, okay,
well, this is what I said I would
most likely do of those options.
Now, what do I think the consequence or
the reaction of the customer is going to
be or the patient or the client or the
guest is going to be on the other end that
allows us to see a little bit further.
So we refer to this as like
service forethought or the
ability to sort of anticipate.
Where does the story go from here?
And so at times we've also referred
to our scenario portions as like a
storyboarding activity, because in
essence, that's what it feels like to the
candidate is there's a little snippet.
Mhm.
Of a story unfolding here, and they're
getting a chance to jump into that
role for a minute, kind of explore it.
And then they move into a different story
that targets a different competency.
Mike Callen: Yeah, that that
is really very much based upon
the response that they choose.
However, they're reacting
to the situation.
So it, it to me seems like it shows
a degree of competence, but then if
you also are able to understand how
someone's going to respond, it, it
probably tends to suggest that they're
going to be better at continuing
that storyboard throughout time.
Yeah, yeah, very, very, very cool.
Chris Cunningham, PhD: And I'll also
note that this, this approach to
testing, I got to remind everybody that
we designed the whole test also be the
kind of an assessment that a candidate
would actually finish because one of the
other problems that plagues assessments
right now is that many individuals will
start, but not everybody will finish.
With our assessments, we generally
see completion rates above 90 percent
and some of our clients is 98%, which
is really high in this industry.
And part of the reason it's so high is
that everything is structured in a way
that it kind of grabs the attention
of the candidate, pulls them in, and
they want to see what happens next.
And before they know it, they're done.
Because the whole assessment, I
mean, it only takes most people
on average less than 20 minutes.
Usually it's around 15 to 20 minutes,
and that's that's a really kind of
focused assessment that yeah, it requires
more than two seconds, but you're also
getting a lot of information out of it.
And the candidates feel it like they
can see the relevance of the questions.
It's not abstract.
It's not way out there
in the ether somewhere.
They can, they can understand why
you want to know this about them
before you make a hiring decision.
Mike Callen: Yeah, absolutely.
And that, that face validity is so
satisfying for for, for, you know,
job seekers versus, you know, the
traditional personality tests.
I think the classic isn't it at the
combine they ask whether the players
prefer hot dogs to hamburgers or some,
some, you know, silly things like that.
Do you like to be in
crowded movie theaters?
Or do you like to go to, you
know, parties where there's
lots of people and loud music?
You know, what does that
have to do with this world?
And there might be some strong
corollaries to behavior, but it
doesn't feel really satisfying.
Whereas this feels incredibly satisfying.
And I think that's such
an important aspect.
Chris Cunningham, PhD: So, you know,
and you bring that up, Mike, and, and
one of the ways we explain this to a lot
of our partners and clients in the past
is, is using a graphic like this, and
we try to help people understand that
when you talk about the, the validity
of assessment, what you're actually
talking about is the ability of that
assessment to inform your ultimate
decisions so that your decisions are good.
So this is what's meant when
people say validity is a, is a,
is a property of inference, right?
It's not actually something
that's baked into the test.
It's what happens after the test.
Well, for that to work, you know,
not only do the candidates have to
appreciate why they're being asked
what they're being asked, because
then they're likely to respond more
honestly and truthfully and completely.
But your hiring managers and recruiters
also have to believe that the test
is measuring something relevant.
Otherwise, they're not going to use it.
You know, and then you also want to
make sure that the test is indeed
measuring what you say it is.
And that's where that cold
construct validation comes in.
And that piece is completely
absent from a lot of the other
assessments on the market.
And then, of course, you need to
be able to show that scores on the
assessment do predict or explain
variability in performance outcomes.
And that's that criterion part.
So what we've done over the years
is we've built a glass totally full
approach here to this challenge.
And we know that that's not.
Necessarily required in all
contexts, and we also know
it's not necessarily the norm.
But again, one of our goals is to try
to help people appreciate the value and
the importance of assessment science.
And that's, that's really been one of
our guiding principles from day one.
Mike Callen: It's very impressive.
We have some experience in the past
with an office and interpersonal
competence test that we created and
we decided to do, you know, very high
fidelity video situational judgment
where we filmed the scenarios and such.
And it was it's, it's been shocking
to realize what the shelf life is on
something like that, you know, you can
spend an awful lot of money creating
something like that only to find
that, you know, the hairstyles and the
fashions change so quickly that it's
maybe the juice isn't necessarily worth
the squeeze to go very high fidelity.
And I think that I would admit to
you that I've tended to be a little
snobbish about the avatar style testing
because to me it always seemed a little
cartoonish in comparison to strict
video situational judgment but it's not.
It's it's not that way at all.
And moreover, I think that
it's abundantly more updatable.
And in fact, you have instances
where you're able to as, as you can
see, in some of these examples here,
you're actually able to customize
certain elements within the scenario
for, you know, marquee clients.
I'm sure that's an upgrade.
But.
But that's really some spectacular
examples of, you know, what,
what it is that you can do here.
So, yeah, thank you for toggling through,
Chris Cunningham, PhD:
yeah, I just want to share.
I mean, and one of the things I want
to share with you is we, we took years
actually figuring out whether we were
going to go video, whether we were going
to use digital art, whether we were not
going to do any illustration at all.
And in addition to the cost
issue that you outlined, Mike,
there's also a speed issue.
You can't update video as fast
as you can update digital art.
There's a major cost issue.
There's also a delivery issue.
So like this assessment platform is meant
to be optimized for mobile delivery,
meaning over a mobile device, which, you
know, in some parts of the world, there's
still not widespread broadband access.
So delivering a video can really slow
things down and create a barrier.
But with these images, you know,
they can pop up and they look
consistently good on any kind of device.
I'll even share one quick
kind of funny story with you.
One of our very first clients, when
we configured the scenarios for them,
they were a larger retail operation.
We got everything set.
They loved it.
And then about a month later,
they did an org change.
And one of the biggest things
that happened is the color of the
uniforms changed from blue to red.
And they're like, we can't have
blue shirts in these images.
Like, Oh my gosh, I think we're
going to have to stop using the
assessment, but we really love it.
We're like, well, wait a minute.
Like, you just need the shirts
to be a different color.
And they're like, yeah,
that's, that's what we need.
And we're like, Okay.
Give us a half an hour.
Like, and that was it was done.
And so it's, it's not hard, you know?
In fact, that's why we built it this
way was to be able to work with,
create partnership with our clients.
Mike Callen: Give us a half an hour
at a hundred thousand dollars and
we'll take care of that for you.
Jenny Arnez: I have to say, I think
this style, this, the, the art
made it easier for me as a test
taker to enter into the story.
I had.
I hadn't taken a Logi-Serve test before.
And so I'm, I'm in there and I
thought, Oh, this is kind of exciting.
I was literally entering in.
Now, if I was watching a video,
I would have felt like an
outsider, just kind of observing.
Definitely.
It felt like I was becoming
part of some adventure.
It was a, it was a remarkable,
positive experience.
Chris Cunningham, PhD: That was actually
one of the inspirations for it was the old
choose your own ending storybooks, right?
Yeah, but the other thing we're doing
here and you know, it's again We're not
trying to trick people but the other
thing you should know about this format
is when we are conveying images like
this or like this or these we're able
to actually not just tell people this
is the kind of environment you'd be
in loosely we're able to render these
exactly like the work environment.
So this particular illustration here this
comes exactly from a client call center.
Like we, we took, this is a high
customization, obviously, but we got, we
went in, we took photos, we rendered it.
So the posters actually
reflect cultural cues.
The technology on the desk is the actual
technology that the person's going to use.
The color scheme is the color scheme.
And so.
And so what's happening is you're
also getting a chance to queue and
send signals to the individual, which
helps improve the relevance of what
they're about to say response wise.
So it's good for the candidate, but
it's ultimately good for the client
because you're getting a higher quality
piece of data coming out the other end.
Jenny Arnez: That's fascinating.
Mike Callen: The video situational
judgment tests that that we did you
know, a big part of the scripted aspect
of it was getting the right camera
angles on the operative person and being
able to express these nonverbal cues
that you want to express, you know,
distaste, displeasure, anger, whatever.
And so that was always something that
we perceived as a huge advantage.
But it's interesting when you're
clicking through the examples that you
have there of some of the different
characters and scenarios, it's very
easy to see these nonverbal cues,
you know, to the same extent that you
would be able to see them in video.
And ironically, I, as I'm thinking
about them, I realized, you know, that
we really, you know, paused on that
person, almost in the same way that that
your static screens are in order to be
able to be sure that we were properly
conveying that that feeling or sense.
And, and so at any rate, again, that's
just, you know, my way of, of really,
you know, complimenting the process
that you folks have invested so much
into because it's very, very effective.
Chris Cunningham, PhD:
Yeah, I appreciate that.
And I know our team does too.
And I will tell you on that last
comment you made, it's interesting.
We really struggled with that for a while
until we realized, listen, you know,
it might be neat to capture the nuance
of nonverbal expression in a video.
But when that gets reduced to the
size of an image on a smartphone,
you're not going to see it.
And so digital art allows us to, and
you might notice like some of our
graphics are, they seem a little like
exaggerated when they're blown up to full
screen, but when they come down to the
level of a graphic on your smartphone,
it looks natural, it looks normal.
And that's, and that was part
of the control that we could get
with this digital art approach.
So yeah, a lot of thought went into it.
I can't, I can't say enough for
our digital art team and their
ability to capture the essence.
Like we would draft items along
with descriptions of here's the
emotions that need to be conveyed.
This is how these emotions are
generally conveyed in a universal way.
Let's capture this in the images.
And we went from there.
Sponsor Break: And we'll be right
back after a word from our sponsor.
Ready to revolutionize your HR strategies?
Head over to testgenius.com to discover
our latest tools and solutions designed
to streamline your hiring processes.
Jenny Arnez: Can you talk a little bit
about how an HR manager or director
might use These assessments in both
pre-hire and then also you've mentioned
post-hire and employee development.
Chris Cunningham, PhD: Sure.
Yeah, sure.
I'm happy to do that.
So I'll start here.
You know, by focusing on competencies,
what we're doing is we're digging
into sort of the foundational
attributes or aspects of a person
that enable them to actually
demonstrate competence on the job.
So in other words, if I if I have these
underlying competencies that I'm more
likely to actually do the right thing
behaviorally in the work environment
and in the service context, the way
this might look in practice is something
like what's represented on the screen
right now, where, if I've got employees
that are competent under an underlying
way, and they're doing the right thing
behaviorally, then that positively
impacts the customer, the guest,
the client, the patient experience.
And that's really good for business.
Okay.
And then the other thing is
when my employees are doing the
right thing behaviorally, it
also improves the operational
efficiency of that organization.
And that has an impact on the bottom line.
So a simple value chain argument like
this helps to illustrate for many
decision makers and stakeholders in an
organization why we want to understand the
underlying competence of our workforce.
From this, we can then expand that
conversation and say, well, listen,
you know you can't probably just hire
top competency people because your,
your labor pool, your, your applicant
flow probably isn't strong enough to
allow you only to take top scores.
So you're going to have more
variability within your team.
Wouldn't it be neat if you could
figure out where they're stronger,
where they're weaker, and then direct
some development initiatives at those
individuals to see if we could strengthen
their overall level of confidence.
And so for post hire applications, that's
often one of the ways this assessment is
used, is we'll generate a report, you get
to see the profile of your individuals,
but you can also aggregate and see group
level sort of summary statistics and
breakdowns of scores, and then you can
work from there to develop training plans.
Here's an example of some of the reporting
that comes out of a typical assessment.
And you can see we've also put
a lot of time and effort into
making this really simple.
So this is not one of those
assessments where, you know, your
users have to go through six months
of training and get certified before
they can do anything with the test.
Like we've actually done studies where
we've gone to HR conferences and we've set
up a little computer lab and we brought
in HR managers and said, Hey, you know
here's here's access to the platform.
Now make a good decision and
justify how you made that
decision in about five minutes.
They can make a good decision because
people understand that gold is better
than silver is better than bronze.
Okay.
But they also understand that from a
medalist perspective, even a gold medalist
has to keep working to stay strong and
a silver medalist has room to grow, but
they're still potentially really valuable.
And a bronze medalist also there might
be worth seriously looking at, especially
if they've got some competencies
that are in a stronger territory.
So we use this type of reporting
methodology also to keep the focus from
day one, not on screen in screen out that
kind of dichotomous decision, but we use
this kind of reporting to always get our
talent managers thinking, okay, this is
what I'm about to potentially employ.
And I want to be able to work with
this individual from day one until
the day they leave the company to
help them reach their full potential.
Why?
Because if I invest in my talent, they
create value for the organization.
So that was always part of the plan,
you know, from the beginning and all
the reporting is designed this way
to kind of facilitate that thinking.
That's also why sometimes this
test is very popular to be used to
identify potential for promotion
because you're looking for people
that are showing maybe growth or
a potential area for achievement.
And then you can put them on a track for
development or some rotational program.
Sometimes it's helpful in that way too.
Mike Callen: Yeah, I, this
is really important aspect.
When we started talking with Eric many,
many years ago about working with you
folks, you know, this is one of the
agreement points that we wanted to make
sure that we shared, which is that when
we're looking at hard skills aspects,
we're looking at skills, abilities,
personal characteristics that are
required day one for success on the job.
So it's very easy to have that
dichotomous situation that you described
where you can clearly have the skills
and abilities necessary for success.
Or maybe you clearly don't.
And so it's really easy to make these
decisions when we're looking at these soft
skills aspects, I love the bronze, silver,
silver because everybody's a winner.
The question is, is what's the
attributes that you do possess at
this point in time that will dovetail
nicely with your bevy of hard skills
that you've already shown to possess.
So I really like the way that these,
you know, two really different aspects
work together towards the same result,
which is recruiting people who are
going to "A" - be successful in the
job and "B" - hopefully continue to be
successful there for many years to come.
Chris Cunningham, PhD: Right, right.
And I'll show you to like
in a more general sense.
This is how that scoring context
gets broken out in most reports.
And so you'll see that our middle
range, our norm, if you match the
benchmark, so to speak, for a given
assessment, then you're going to land
squarely in the silver territory.
But if you see a gold medalist or a
high silver person, somebody has a score
above 500, that's likely a very strong
candidate with respect to that competency.
And so the, the, the gist or the guidance
for a user is quite simple, like aim high.
Okay, but don't feel like you have to
wait for all of your applicants to test
so you can rank order them at the end
of the process because we know that's
not feasible for most organizations.
So instead, you know, look for people that
land in that silver and gold territory
and give them serious consideration.
And if your applicant pool is not strong
enough, and all you end up with is a
lot of bronzes, but they might have some
strengths in the silver gold area take
a close look at them and ask yourself,
are we prepared to work on building
some of these bronze competencies?
Because if we are, then it might
be a really great candidate.
Mike Callen: Yeah.
And again, especially if you have
other, if they possess other hard
skills, at a very high level.
And you know, there are certain tests
that help to measure the trainability
aspect, you know, your comprehension
and those kinds of aspects.
And so if you have somebody, you know,
who like in your example is a high
bronze, has good skills and abilities
appears to be trainable, but would be
a great addition to your team then it
makes it a lot easier to take a chance
or, or maybe dip the ladle a little
bit lower into the different tiers
that you have among your candidates.
Chris Cunningham, PhD: Absolutely.
Absolutely.
And, and the one thing I just wanted
to add with this particular graphic,
I'll just throw it up on screen here,
is you'll you'll notice and this
actually comes from an actual client.
And so this is a representation of
how an assess how this assessment
can differentiate between strong
performers and everybody else.
And you'll notice that it's not
always the case that all nine
competencies are equally influential
in determining who a strong player is.
So that's another piece that has to
be taken into account for any given
client any given users Like maybe they
did have a bronze in this example here.
Maybe they had a bronze in proactivity,
which is down the lower left corner.
Well, you know what?
That was not a strong
differentiation point.
So I wouldn't lose sleep over that.
On the other hand, if, if we have an
individual with a bronze and motivation
and interest, which is just to the
right of proactivity, you notice
the gap between the top performers
and everybody else is massive there.
I would be really worried at that point.
If, if they were bronze medalists,
I only want silver and gold when it
comes to motivation and interest.
So my point is people using the assessment
have a pretty good understanding in
their organization of what what is
critical, what is not so critical.
And the beauty of this assessment is
it gives you that ability to make that
more nuanced decision because it isn't
pass fail like you're saying, like, but
it does give you guide rails, right?
And it makes it pretty easier,
easy for you to kind of figure out.
Okay, what is the balance trade off risk?
We're willing to take with this candidate.
Jenny Arnez: So are you recommending
that HR people, managers, they look at
these nine competencies and they, they
rank them as to what's most important
to them and for a given position?
Chris Cunningham, PhD: So, sometimes
that's helpful approach but I'll
show you again in the main report.
The, the number that most
people use is the overall.
That's the one that's at
the top of the report.
And what that is, is an
aggregation of all nine.
And the, it's, it's beyond our
scope today to talk about exactly
how these scores are generated.
But I will say to you that these
scores are, there are individual
scores for every competency.
And there are composites that combine
them in certain weighted fashions,
and then there's an overall composite.
And by and large, most of our users
first look at the overall score to
do some initial filtering, and then
they go down to the nuanced breakout.
And you can even see in this mock up
here on the screen, this particular
candidate is not a strong player.
But they happen to be okay, a
little okay with proactivity.
So I could look at this and say, well, you
know, I, I would consider this a higher
risk person, but if they, if they have
productivity, then that's a really great
attribute, especially if I'm willing and
able to invest a little bit in helping
them see gaps that they need to address.
And so I don't know that I
personally would take a risk on
this particular candidate, but if I
got into a bad situation, I might.
Mike Callen: Yeah, you, it gives
you the awareness of the risk.
So if you're in a situation where,
you know, you've it's a little crude,
but a lot of the people that we work
with talk about butts in the seats.
You got to get butts in the seats.
And you know, sometimes you need to
have a human being in there and it
may not be your first or second or
third choice, but you've got to get
somebody in there doing the work.
And so you have a good idea of what
type of efforts going to be necessary
in order to, to bring them along.
You know, to that end, Jenny
asked you about post hire.
And I know that for a lot of positions
there are preceptor or trainee.
Or sorry, trainer to trainee
relationships, and there you go.
That's one of your aspects here.
Do you find, or do you recommend that
having the preceptor or trainer go through
take the test and then trying to, you
know, match behavioral types between
those would make the training slash
onboarding process more successful because
you're matching like people together?
Or is there any research
that you've done on that?
Chris Cunningham, PhD: I am following
what you're outlining, and I think
there probably is some evidence that
that approach might help, especially
if the, the training and development
model is, is kind of a mentoring
forum where you're trying to match
people with you know, with, with
that sort of mindset and approach.
I can't tell you that we've done
research on that particular model
yet, because most of our clients.
Have wanted to use this to
help make better decisions
about new talent or talent.
They want to develop.
There's some that have started using
the assessment to guide promotion
decisions, but we haven't seen
a ton of interest in using it to
pair partner employees together.
So I think it's, I think it's a
great illustration of another way
in which this competency approach is
so flexible and is really designed
to support good talent management.
It's not simply a pre hire
selection tool like that.
That's one point I think we just
need to emphasize is that there's
a lot of opportunity here to
use this both pre and post hire.
And that was by design.
So we'd really love to see clients
use it for all of its richness.
Mike Callen: Excellent.
Well, you know, if that comes up down
the road, then you know, maybe that'll
be a great opportunity to have you back
and talk a little bit more about that.
I know that, like in nursing, for
instance, that preceptor trainee
relationship is super important.
And so you know, because people do
learn differently, there could be
a benefit and, you know, sort of
matching them up style to style.
But at any rate, it was just something
that was that I was curious about.
Jenny, do you have anything
that you wanted to bring up now?
Jenny Arnez: I I just a
general, just a comment.
Taking the assessment, the whole
approach was the most positive
assessment I've ever taken.
It typically, you know, when you go to
take a test or an assessment in, say,
Excel, it's a little nerve wracking.
You can, you can get a little bit
stressed, but I didn't feel that in this.
And so kudos to to you all for
developing something like this.
It's amazing.
Chris Cunningham, PhD: I appreciate that.
And I don't have a graphic to add here,
but I, I will share with you because
we've actually done some direct testing
of candidate experience and reactions and
a lot of times what we hear is that more
than 90 percent of our applicants describe
the assessment as fun, and 90 percent or
more actually describe it as easy, and
people have tried to tell me at times that
an assessment shouldn't be easy and then
I say to them, how's that working for you?
Because the thing is, like, if
our goal is to trip and trick and,
you know, make people feel stupid.
Then as an industry,
we've done pretty great.
But if our goal is to learn about
people and make better decisions so
that fit is ultimately accurate, then
we need an assessment that people
are going to be willing to take and
actually give their best selves too.
So, so I think that that's
really really helpful.
The other interesting metric I would just
share with you all is we've also tracked
sort of a net promoter effect with this
assessment and in some of the engagements
we've had with clients, we've asked
people, "Hey, you know, now that you've
had this assessment experience how likely
would you be to recommend a friend of
yours to apply for a job here as well?"
And again, more than 90 percent of users
would recommend that company because
of their assessment related experience.
And I don't think we spent enough time
talking about that in this industry
because you gotta understand like if
we create a front end experience that's
abysmal we're also damaging our ability
to recruit talent because thanks to
the power of the internet, right?
Like if one person has a bad experience,
the rest of the world knows about it.
So having a good front end experience like
this, having a good internal experience
like this, I think it really matters
now more than it probably ever had.
Mike Callen: We do at least 50 percent
of our business in the 911 emergency
services, police, fire dispatch space.
It's a huge market for us.
And in this post COVID world it's
been uncanny how every conference
that we go to, there are big sessions
that are led by roundtable groups
of people who are talking about
how do we bring more people in?
How can we improve our situation
so that we get, you know, a little
bit of momentum going in our,
communities, you know, towards, you
know, bringing more people into here?
It's, it's become a more and
more challenging situation.
And you know, I, I do tend to kind of
come, you know, from the side where at
least on the hard skills we've got to be
able to ask them some tough questions.
And, and to a certain extent, you know,
if you can't like to, to use Jenny's
example, if you can't work Excel to a
very high degree and the job requires
you to work Excel to a very high degree,
then, you know, you shouldn't feel
as though you, you passed the test
with flying colors if you failed it.
But when you're dealing in the, this
behavioral soft skills realm, you know,
that's really kind of an opportunity.
People are what they are, you know, to
a great extent at this point in time.
And so to have an opportunity to really
sort of feature yourself and, and do
so in a way that that is affirming and
positive is, is really is really great.
And, and actually what it sort of leads
me to think might be a good best practice
when we're putting tests together within
within our platform, including your
tests, you know, that we may want to, to
recommend to people that they go through
and they put the hard skills tests first
and let people finish with something
that's a little more fun and is going
to leave them with maybe a little bit
better memory of the testing process.
Chris Cunningham, PhD: So yeah,
yeah, that's something that would be
interesting to explore because you know,
the other the other approach is to warm
them up with something easy, right?
And they get them excited
about well, okay, they've
asked me good questions so far.
So let's let's go a little further.
But, you know, what I think is
important here to to point out
is, you know, worrying about the
look, the feel, the experience, the
assessment, this is not like meaningless
conversation for an academic audience.
Like, we got to also remember that
unfortunately, assessments have also
had a pretty bad history of maybe
discriminating against certain groups
and not really leveling the playing
field as much as they're supposed to.
And that's something else that
we've spent a lot of time working
with in the development of our
tool especially is making sure that
it does level the playing field.
And we've actually seen some really strong
evidence of this in a number of studies.
And what I mean by that is that this test
does not show preference to any group.
So if you're a member of a minority or
protected group, you have just as much
chance of scoring well on this as somebody
who's a member of a majority group.
It's not a cognitive test, first of all,
okay, and it's focused on competencies
that you could have developed in a variety
of different life domains and experiences.
Why I'm emphasizing this is that we really
do, as an industry, need to do more to
think about how we package together these
screening experiences and really just
keep asking ourselves, like, are we asking
questions that that get at attributes
that need to be present day one, week
one, or are we focusing on things
that we can actually build over time?
And I know this is a, is a hard challenge,
especially when you're working with
skills testing, but I know you guys
have done a lot to tease that apart
with your clients and make sure they
understand that when they're putting
these assessments in place as well.
Mike Callen: Definitely.
We have we haven't talked much about
the structure of your assessments in
terms of the library itself, but you
have a sort of a service focus, and then
there's a sales focus, and then there's
a managerial leadership potential focus.
And then you have many different
industries or verticals that
are represented with each
of those particular areas.
And then you can drill down further
and you can choose specific job titles
that are very standard across the
country, or maybe even around the world.
All told I think that we have about
shoot 220 or so of the Logi-Serve
tests available in our library
to our clients who have that
particular add on service activated.
So the, the new almost every new
client that comes on board since
the integration has been done will
be getting this particular service.
And then those clients who signed on
prior to it have the opportunity to
upgrade their subscription to include it.
But it's a very, very robust
library that you've you've made
available and very impressive.
Did I leave out any of the
details about the library or
anything that you'd like to add?
Chris Cunningham, PhD: No, I think
that's a good summary of an explanation
of the value add that this brings.
And you all already have a wonderful
product that can really help
companies make better decisions.
And we're really happy and excited to
become a partner that can help you take
it to another level with your clients.
We've we've seen a lot of success
using this product as a standalone.
So we know that when it's partnered
with another well designed product,
the impact will be even greater.
And we're just excited to kind of
see some of these impacts happen.
Mike Callen: Excellent.
As are we for sure.
One of the things, areas that
I reflect upon is the,
the leadership side of it.
And I don't know if you have any
particular insights on this, but
having grown up in, in sales very
often organizations will, you
know, promote their best performing
salespeople to sales managers.
And sometimes that can have
good results and sometimes
it can kind of be disastrous.
Sometimes the best salespeople
aren't the best sales leaders.
And sometimes the best sales leaders
aren't the best sales people either.
So being able to add something to
the conversation that gives you some
insight and, and intelligence into,
you know, deciding who it is that
you promote into these situations
can be, can be very, very helpful.
And some of the smaller organizations
of which you know, most of the
businesses in America are you know,
making the wrong decision in this
regard, you know, especially for
instance, promoting a really good
sales person to sales management and
having them fail can be disastrous
to, to that particular organization.
Chris Cunningham, PhD:
Yeah, yeah, that's so true.
And, you know, to add a couple of
layers on this, I mean when you take
a very strong individual contributor
like the examples you've been giving
Mike and you put them into a managerial
role, even if they are effective as a
manager, you've lost the value of their
performance as an individual contributor.
So it's a real big gamble and we have
to be careful because sometimes the
characteristics and attributes that make
individuals powerful as an individual
can actually hinder them from being
good at working well to support others.
It's, it's a different competency
set to be a leader and a manager
than it is to be a sole contributor.
So that's exactly the rationale behind
why we built that, that vertical out.
And you know, in that particular,
assessment we've also seen it be used
a lot to help organizations increase
diversity and their leadership potential
and their leadership development programs.
Because the problem with the model
that you shared before, Mike, that
is so common, is when we get in the
habit of identifying our current top
performers or we ask managers to, you
know, identify talent, it tends to
happen through a similar to me process.
Like, you know, the manager
says, Oh, well, yeah, Joe reminds
me of myself 20 years ago.
I think He's the one to tap.
And of course, the problem
with that is that that
automatically creates homogeneity.
And it really damages an organization's
abilities to go in a different
direction or to really open that
door for more people to consider it.
So I mentioned leveling
the playing field before.
It, in leadership, we see that as a really
big need in many organizations because
they just don't have a process in place
that's scalable to figure out who do
we want to fast track for development?
Who do we want to put on that
leadership track so that we're
ready when those roles open up?
Mike Callen: Yes, very, very good points.
Well, I, Jenny, don't
have any other questions.
I'm looking through my, my list
of reminders that I had that
I wanted to ask Chris about.
So, Jenny, do you have any other
questions that you'd like to ask
or comments you'd like to make?
Jenny Arnez: No, I don't actually, I feel
like we're kind of we're winding down.
We're at a good stopping point.
I think for this conversation,
I mean, I definitely have other
thoughts that I'd like to ask you
sometime Chris about AI and we kind
of touched on this in a previous
conversation, but that's not for today.
That's a, that's a whole different, street
that we'd be driving down, but I, I just
am so grateful that you joined us today.
And I've thoroughly
enjoyed this conversation.
Any final thoughts from
either one of you guys?
Chris Cunningham, PhD: I'm just
appreciative we have the chance to
have these types of conversations.
A lot of times what holds other
clients back is lack of awareness
of some of the complexities and
nuances that operate under the hood.
And then just being able to talk about
some of these topics, it may not be
something that was on somebody's mind,
but then when they learn about it, they're
like, yeah, you know, that was on my mind.
I just didn't know what to call it.
So it's important to have these
types of discussions and to
raise these types of issues.
So thank you both for
putting this together.
And I'm always happy to, to
take that trip down those other
streets when you guys are ready,
Mike Callen: That's great.
Yeah.
I, I, I really appreciate you, you
being here today and you know, we,
we hope that this content, you know,
meets the needs of new HR people.
People are getting established and really
trying to learn, you know, about different
aspects of testing and selection.
But like I told Jenny before we
started today, I was really excited
about this because, you know,
it's fun to geek out a little bit.
And you know, when you have, you have
somebody that's a chief science officer,
I mean, it sounds like you, you started
in Star Trek and, you know, you've just
kind of graduated to where you are right
now, but it's, it's really great to to
be able to pick your brain and and hear
your opinion on a lot of these things.
As I mentioned, we've been
in conversations with Eric
for many, many years now.
When we first started talking about
integrating our products, you know, we
had an installed product and it wasn't
something we could do at that time.
But we've persisted in the conversation.
And so, you know, now we're here.
We've launched we've got the generic
off the shelf products available.
We're going to work together to create
a market killer product for, you know,
the 911 EMS police and fire space.
And we're super excited
about that as well.
And so anybody who's listening to this if
you're interested, I'm sure we'll have a
form or, or some sort of call to action
available at the end of the video here.
If you're an existing client and
you'd like to, you know, upgrade
and add this incredible line of
testing to your subscription.
You can reach out to us and
we'll be happy to get you set up.
We can literally get that launched
for you in about five or 10 minutes.
And if you're not a client and you'd
like to talk more about being one and
being able to take advantage of this
incredible Logi-Serve product along
with our hard skills tests we'd love
to chat with you about that as well.
Again, Chris, thank you so much for
being so generous with your time today.
We really appreciate you
you know, sharing with us.
So thank you very much for that.
Chris Cunningham, PhD: Likewise.
Jenny Arnez: And we just want to say
thank you to our viewer and our listener.
We're sure glad you joined us today and we
hope to connect in another place and time.
Thank you everybody.
Mike Callen: Thank you.
Podcast-Outro: Thanks for tuning in to
Testing Testing 123 brought to you by
TestGenius and Biddle Consulting Group.
Visit our website at testgenius.com
for more information.