HR Tests - Recruitment, assessment, and personnel selection

Nice post! i am glad to hear about new efforts to ...

2009-10-29T08:16:01.524-07:00

Nice post! i am glad to hear about new efforts to validate personality assessments for personnel selection. I have yet to read the article, but I am a bit surprised to see a significant relationship between conscientiousness and task performance. Granted, an R-Squared of 5% doesn't seem very big, but considering were aren't talking about contextual performance, I am pleasantly surprised.

First of all. Thanks very much for your useful pos...

2009-10-22T07:31:46.843-07:00

First of all. Thanks very much for your useful post.

I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.

Please let me introduce you some info related to this post and I hope that it is useful for community.

Source: job analysis sample

Thanks again
Ngo

haha! Also, i don't think we use the term "R-Squar...

2009-10-06T15:29:38.152-07:00

haha! Also, i don't think we use the term "R-Squared" enough. People love that!

Good point! It's probably all those correlation ta...

2009-09-30T18:17:11.853-07:00

Good point! It's probably all those correlation tables...

Thanks for the positive feedback Tasha!

2009-09-30T18:13:45.101-07:00

Thanks for the positive feedback Tasha!

Haha! That's a fool's bet. I suspect SIOP will ret...

2009-09-30T15:58:51.175-07:00

Haha! That's a fool's bet. I suspect SIOP will retain its name. I am not even sure how I feel about the issue. I suspect that our name is not our largest obstacle in letting people know the value we can bring to the workplace.

My boss attended your session in Nashville, and is...

2009-09-30T11:43:35.282-07:00

My boss attended your session in Nashville, and is touting it as the best one he saw. Also, just wanted to say what a nice job you did on the web conference yesterday. I just happened onto your blog from the IPAC site and I'm enjoying it.

I too get very nervous when organizations use pers...

2009-09-23T12:48:15.925-07:00

I too get very nervous when organizations use personality assessments for selection purposes, or even during their selection process. We certainly have some evidence to suggest that some elements of personality can predict job performance, but the conclusions are not universal, nor are they universally understood.

It is way too easy to get this wrong. All too often personality data tells us something different than we intuit. I simply do not trust the usual decision makers to limit their decisions to conclusions supported by the data. Check out this post where i talk about a similar issue: http://wp.me/pycJE-X

CTRL+X: you raise some great issues, very well sai...

2009-08-07T06:24:28.926-07:00

CTRL+X: you raise some great issues, very well said. I think reasonable people are going to disagree on the impact of this case and we really won't know what the impact will be until we start seeing more cases. An example is the New York City firefighters case, where the judge mentions Ricci but distinguishes it just the way I would predict by pointing out that Ricci is (as you mention) a disparate treatment case, not an adverse impact case.

If in future adverse impact cases an employer attempts to cite Ricci as setting a lower bar for meeting the "job related and consistent with business necessity" burden, as a plaintiff I would point out that Ricci dealt only with the case of defending one type of discrimination by avoiding another. Yes, the court made some (frankly, kind of odd) comments about adverse impact liability, but it does not change the burden shifting under Title VII. Failing to produce evidence to overcome a "strong basis in evidence" standard is not the same as meeting your burden under a typical adverse impact case.

I think we need to remember how unusual of a case this was. It's very rare--and I would assume will continue to be--for an employer to argue against its own test.

Although I agree with some of your points, I belie...

2009-08-06T11:04:26.754-07:00

Although I agree with some of your points, I believe you minimize the potential impact of this case. The Court’s analysis was of the "premise that the City’s actions would violate Title VII’s disparate-treatment prohibition absent some valid defense." However, they evaluated this disparate treatment by looking at issues of disparate impact. So, they essentially considered the merits of a non-existent case without the input of individuals or experts who would have had a “say,” if that case would have existed. Issues such as the job-relatedness of test weights and consideration of alternatives were dismissed as legitimate issues and presented as if the city had somehow met the requirements of the UGESP in both of those areas. If this case would have actually occurred as a disparate impact case with Black firefighters as plaintiffs, those issues would have been presented with significant focus and extensive expert testimony. However, the Court overreached on their conclusions of these issues, which unfortunately has the potential to set precedent regarding these types of considerations in future disparate impact cases. The Court could have (and probably should have) ruled that the City's arguments (i.e., those validity issues) were pretext to their decision to scrap the test and subsequently a disparate treatment violation. However, they went beyond and stated that those issues are not only pretext, but also not legitimate. For example, here is a quote from p. 30 of the Opinion:

But respondents [i.e., the City] have produced no evidence to show that the 60/40 weighting was indeed arbitrary. In fact, because that formula was the result of a union-negotiated collective-bargaining agreement, we presume the parties negotiated that weighting for a rational reason. Nor does the record contain any evidence that the 30/70 weighting would be an equally valid way to determine whether candidates possess the proper mix of job knowledge and situational skills to earn promotions.

All of these statements regarding production of evidence are completely contrary to the burden of proof that would have been in place for a disparate impact case. The plaintiff would not have to prove that the weights were "indeed arbitrary", but rather that they were not job related and that other equally or more valid weights were available and would have been expected to minimize AI. On the otherhand, the City would have to provide evidence that the weighting was job related, instead of "rational"...not to mention the issue of "equally valid."
On the whole this decision seems to muddy the water with regard to disparate impact litigation by considering evidence in a manner that is inconsistent with the UGESP.

"What would you do if you saw a £/$ 20 note lying ...

2009-07-27T00:16:37.072-07:00

"What would you do if you saw a £/$ 20 note lying in the street?"

All candidates would say they would pick it up and hand it to a policeman but isn't that what they want you to think they would do?

Most people would probably think
"wooo hooo" and go straight off and spend it...

Thank you, that is an important distinction. I ch...

2009-07-26T11:49:25.603-07:00

Thank you, that is an important distinction. I changed the post to reflect.

Bryan - Important note -- the Amicus Brief that ...

2009-07-26T11:43:02.112-07:00

Bryan -

Important note -- the Amicus Brief that was written by Aguinis and others was not written to represent SIOP or at the request of SIOP. Those authors all happen to be SIOP Fellows, and certainly respected for their opinions and experience in employment testing. But an important caveat to that Brief is that it is not meant as a representation of SIOP's opinion nor to represent the SIOP membership's perspective on the case.

I think that is an important clarification given the contents of the Brief.

I'm happy to share my slides, and always open to f...

2009-07-25T15:21:21.161-07:00

I'm happy to share my slides, and always open to feedback.

As far as your quote from the UGESP and observation, I agree with you. Particularly over thr last 10 years, the SCOTUS has made it clear disparate treatment is only legal in rare cases and usually only temporarily.

I'm still not seeing how this case changes things in any dramatic fashion, regardless of how much I/O psychs nash their teeth. I guess we'll see!

Bryan--Any chance you'll post your slides from the...

2009-07-24T07:12:49.955-07:00

Bryan--Any chance you'll post your slides from the August luncheon presentation?

Here's a quote from the Uniform Guidelines - Q & A 28:

"Whether or not there is adverse impact, Federal equal employment opportunity law prohibits any deliberate discrimination or disparate treatment on grounds of religion or national origin, as well as on grounds of sex, color, or race."

Is this not the essence of the Ricci case? Intentional discrimination is rarely, if ever, justified--even if done to avoid what is believed to be unintentional discrimination.

Well that would certainly be refreshing! And most...

2009-07-19T06:45:24.441-07:00

Well that would certainly be refreshing! And most likely narrow down the candidate pool considerably.

All good attributes described. How about this one....

2009-07-18T15:09:01.837-07:00

All good attributes described. How about this one. Executive agrees to an incentive plan that is actually hinged to performance. Executive understands that if he or she drives the company into the ground they don't get a zillion dollar as they are run out of town for a job not too well done.

Bryan—I think we’re on the same wavelength. A big ...

2009-07-08T06:37:47.618-07:00

Bryan—I think we’re on the same wavelength. A big fuss was made about this case but when the decision came down it repeated what we’ve been hearing for some time: disparate treatment (intentional discrimination) is bad, disparate impact (unintentional discrimination or adverse impact) is bad, and oh by the way, committing one type of discrimination to avoid the effects of another is also bad. Like we didn’t know that already!

Great insight! EQ is really getting interests from...

2009-06-25T22:43:13.090-07:00

Great insight! EQ is really getting interests from the recruiters now, the smarter ones at least, because of the reasons you mentioned. Also behaviors, at times, are more important than mere skills specially in large companies with open organizational culture. For this purpose, at HireLabs we have included behavioral aspect in our customized tests. For details, please visit http://www.hirelabs.com

I completely agree! Interview has importance but i...

2009-06-25T22:02:04.979-07:00

I completely agree! Interview has importance but it should be a final step not a base for selection.
In fact, I believe the first step towards any recruitment is proper assessment according to the desgination and organization. As at HireLabs (http://www.hirelabs.com) we make customized tests according to individual organization and job requirement. It really helps and fast companies are adopting it quickly!

Very interesting issue has been raised. Well, in m...

2009-06-24T05:01:32.439-07:00

Very interesting issue has been raised. Well, in my opinion the vendor size is relevant but it is not a make or break issue. I think product quality is much more important than the size.
With good assessment techniques even the small companies can solve their succession planning issues like we do at HireLabs. We use latest pre-employment techniques to have people who, in the first place, rarely leave us and if they do we can hire the next with the appropriate fit with our organization. Leaving no room for such issues. Thanks to our latest testing and evaluation techniques!

This post pointed out some things I haven't given ...

2009-06-22T11:40:54.278-07:00

This post pointed out some things I haven't given much thought to. In the past I have applied for several jobs that use a PEO to administer assessment tests. In these tests I have been asked questions ranging from what kind of experience I have to personality. Now when I think about these past tests, there were questions that asked how much work load you like, how much stress can you handle, etc. Those questions gave the employers a better idea if the job is right for you, however. The jobs that I have left were not because of the job itself. It was true to the test questions in workload, stress levels, etc. It was for personal reasons for leaving the job. In some cases I left because of other employees, and other cases I left because the job just wasn't what I thought it would be. In other words- misleading. Maybe some assessment tests should start to focus more on personality assessment and go more in depth.

Good question. I'd like to say that in most cases...

2009-06-15T20:48:59.961-07:00

Good question. I'd like to say that in most cases there is some sort of logical (mathematical) combination based on job analysis information. But in most cases the interview probably trumps the work sample--particularly if the interview doesn't go well. Of course that may depend on the order of the selection process.

You are right on about this. Cognitive ability tes...

2009-06-15T14:34:42.666-07:00

You are right on about this. Cognitive ability tests are great, but provide general information. Work samples are fantastic supplements for selection. Of course, this means that selection agent must have a clear grasp of the job they are selecting for (not always a given).

Further the job sample should be validated. Perhaps the biggest obstacle for the use of these test is the insistence that an interview be used. To the extent that the information gleaned from interviews counter-indicates the information gleaned from other tests, how will the typical selection agent synthesize the information?

My team has been researching EQ for over a decade....

2009-06-11T07:51:06.261-07:00

My team has been researching EQ for over a decade. In 1997, we developed an EQ assessment that was broken down into a number of factors. The factor structure has changed over the years as we gained more and more insight, but essentially, we always reported two subfactors - 1) Self-reported traits and attitudes, and 2) Knowledge component (assessed by situational questions).

Our EIQ Test assesses ability to identify emotions and emotional insight (into self and others), ability to express and manage emotions, social insight and empathy, goal-orientation and motivation, and integration of emotions. The Knowledge scale focuses the individual’s ability to recognize or identify the responses (emotional, cognitive and behavioral) that in the given situation present the most effective way of achieving the desired outcome, whether it’s dissipating an explosive situation, getting others to go along with what you want, or soothing yourself or others. The scale also assesses how well you can read a situation from overt behaviors of others.

The correlations between the self-report and knowledge components in our studies is in the 0.35 range (p < 0.0001), which is higher than the one reported in this blog, but it still goes to show that what people say and what they do is not the same thing.

There are several reasons why, in general, self-reports are less reliable – impression management is a big one, especially in a pre-employment testing situation. Self-deception and true cluelessness (or lack of self-awareness) about one’s ability to read others (and self, for that matter) are also likely to contribute to the “noise” found in assessments using self-reports.

The bottom line is: the ability to figure what others are feeling and adjust one’s behavior and communication style accordingly is an extremely valuable skill for anybody who deals with people, regardless of whether that means working with clients, teammates, direct reports or management. As such, it should be assessed, developed and used in performance evaluations.

Ilona Jerabek, PhD
PsychTests.com