HR Tests - Recruitment, assessment, and personnel selection

Research update

2013-05-19T09:28:00.001-07:00

Okay, I'm just gonna say it. There is an insane amount of research out there directly or indirectly related to our field. I'll be honest, it's a little daunting thinking about reading and processing all of it.

Luckily, you have me to overly simplify it for you so you can incorrectly describe it to others. So let's jump right in!

First up, the June issue of IJSA:

- Fairness perceptions matter in web-based selection too!

- The predictive validity of conscientiousness is moderated by self-enhancement

- Speaking of conscientiousness, it's a better predictor of performance in routinized jobs than those with complex cognitive requirements

- Psychological hardiness predicts adaptability in military leaders. This study is awesome also for showing a negative relationship between SAT scores and West Point performance.

- Recruiters, listen up: this study found that contingent workers converted to full-time status performed as well as referral and online hires, but not as well as those sourced internally (bookmark this one, there's a dearth of recruitment research)

- Scoring biodata: empirical keying reigns supreme over rational and quasi-rational approaches. Quasi-rational? Sounds like me most of the time.

- Personality and job performance: interactions are important beyond main effects

Okay, next, the May issue of JAP:

- Adaptive decision making in military leaders: both the brain and the mind are important (okay you monists out there, chill out)

- Are you a MANOVA (wo)man? Then read this.

- Proactive leaders set more challenging goals and have higher sales performance--assuming they have the trust of their subordinates.

Next, the April issue of JOB. Just one study, albeit interesting, in which female evaluators were less likely to recommend hiring or promoting Asian (versus White) applicants into jobs requiring social skills

Speaking of JOB, how about the May issue?

- Have you been calling for more research on calling? Your wish is granted.

- Curious about the concept of curiosity? (okay, I'll stop) Looks like it can be predictive of job performance above and beyond traditional cognitive and non-cognitive predictors. I'm gonna assume it varies with job, but I think the authors are probably right that it will increase in importance over time.

- When selecting for teams that may experience crisis situations (e.g., nuclear power plant crews), consider homogeneity--not mean levels--of positive affect.

- Moderate levels of supervisory structure combined with high levels of consideration lead to lowest levels of CWB's

- The dark side of OCBs

- The light side of OCBs

- Dark side and light side working together in a picture that has nothing to do with selection

Okay, onwards and upwards: one from the April issue of JASP, on generalized self-efficacy, work-related self-efficacy, and job-related outcomes

And one from the May issue: Sensation seeking and need for structure predict military field exercise performance

How about a little EI research? Haven't had a lot of that lately. Here's a piece from the May issue of Journal of Management that found emotion management ability to be a valid predictor of job performance.

Let's look in the May Psychological Science:

- Where we find a fascinating study that demonstrates assessment of profound cognitive abilities at a young age predicts outstanding contributions in many adults

- Another interesting study of how an individual responds to daily stressors (which I would posit are differentially found in various jobs) has long-term consequences for their mental health

- High schoolers with high math and verbal skills are less likely to choose STEM careers than those with math skills but moderate verbal skills

- Okay, check this one out: mindfulness training improved GRE reading comp scores as well as working memory. Implications for personnel assessment??

Shifting to the May Psychological Bulletin:

- Rorschach, anyone?

- Believing that individuals are malleable, rather than fixed ("implicit theory") predicts goal achievement. Implications for leaders?

Last but not least, let's hook up with our friends at PARE and see what they're up to:

- Into Excel? Check out this article on forest plots

- Or maybe factor analysis is your thing.

That's all folks! Hope your brain is spinning like mine!

Test validity: A fairy tale

2013-04-27T12:20:00.002-07:00

Once upon a time there was a field called Industrial and Organizational Psychology. Its researchers and practitioners dealt with a myriad of magical issues ranging from individual differences and behavior to organizational structures.

Within this field, there was a specialty called Personnel Psychology. It dealt with narrower--but no less mysterious--issues such as defining and designing jobs and, most relevant for us, finding and hiring the right people. The I/O psychologists and HR professionals that quested for these answers often found themselves on dangerous missions like battling Monsters of Doubt (i.e., first-line supervisors).

These adventurers had two main weapons at their disposal when fighting these monsters: the Carrot of Truth and the Stick of Pain.

When invoked, the Carrot of Truth, fashioned deep in the Mines of Correlation, caused monsters to realize that hiring the right people was the right thing to do for their realm. It increased productivity and morale, customer satisfaction, and organizational flexibility. It also allowed supervisors to spend more time leading and less time dealing with gremlins (i.e., employees with performance problems).

The Stick of Pain used an opposing form of magic but was sometimes equally effective. It attempted to slay the monsters using a peculiar power called The Law. The Law frightened monsters because it meant they could experience emotional pain and suffering, and--more importantly--fewer bags of gold.

For a long time, Personnel adventurers used both of these weapons to slay all kinds of monsters, on high mountains and in dungeons. But over time, the adventurers discovered something: the Stick of Pain was becoming less and less effective.

It wasn't that the Stick was powerless. It's just that its magic didn't seem to frighten the monsters as much. The monsters saw their gold piling up and didn't feel the sting of suffering as they once had. And they started developing an addiction to carrots all on their own.

So there came a day where the adventurers and the monsters met on the battlefield and came to an agreement. No longer would the adventurer wield the Stick of Pain. And in return, the monsters pledged to respect the Carrot of Truth. They forged an eternal partnership and lived happily ever after.

The End.

Okay, so I've taken a little artistic license with my blog post today. But hopefully you see where I'm going.

Back in the old days (ya know, like the 80s), employers were faced with a foreboding world of testing, with the Civil Rights Act and cases like Griggs vs. Duke Power looming large over their assessment programs. I/O psychologists were brought in to help organizations navigate the complicated world of employment testing, which required an appreciation of statistics and the law alike. Large awards and settlements brought C-level attention, and regulatory agencies like the EEOC and DOJ served in ongoing oversight roles, requiring employers to clean up their act with procedural requirements that could be burdensome.

Nowadays, I/O psychologists are as likely to be valued for their ability to crunch "big data" to detect employee behavior trends as their ability to conduct thorough job analyses (not that the two are mutually exclusive). Lawsuits regarding testing are infrequent compared to issues like wages and hours, harassment, and terminations. The selection cases that do come up are as likely to involve disabilities as adverse impact due to cognitive loading.

Sure, we have the occasional big case that gets attention. But the bottom line is over the years the "stick" has become much less effective as an argument for sound assessment than the "carrot."

Smart employers like Google have started crunching the numbers and realized the true business value of defining the right competencies for jobs. They're doing so not because they're afraid of litigation, but because they see more clearly the direct line between best practices in selection that we've been preaching for years--i.e., focusing on valid assessment results--and the bottom line.

So where does that leave the stick (i.e., fear of lawsuits)? Is it time to put it away along with phrenology and T&Es (woops, that slipped in)?

I don't think so. Organizations will always be subject to legal scrutiny when their selection processes have adverse impact and the right person talks to the right attorney. Personnel psychologists and HR professionals should always have a healthy respect for the legal climate we operate in, and not forget that "job related and consistent with business necessity" isn't fictional gibberish.

But what it does mean is that because organizations are paying attention to their assessments, they are more likely to yield valid results and be more free of illegal bias. That means management's quest and the selection professional's quest are more likely to converge, with a lot more cooperation.

And hopefully a lot fewer monsters.

Presentations from 2013 ATP Conference

2013-02-23T08:25:00.000-08:00

Many of the presentations from the 2013 Association of Test Publishers conference are now available and I strongly recommend checking them out. Strong themes include gamification, global assessment, item development, and test security, just to name a few.

I found the one from HumRRO on creating a virtual role-play fascinating. If you haven't already seen it, I strongly recommend you check out their demo at www.humrro.org/simdemo.html. The username and password are both humrrodemo. A small taste of what's possible in assessment!

Hat tip.

Research update addendum

2013-02-18T07:00:00.000-08:00

Okay, I know I just did a research update but but I had a couple stragglers, including a pretty important one, namely the Spring 2013 Personnel Psychology, which is free right now!

- First, an important piece by Bobko and Roth updating d values for Black-White differences on selection measures. Their updated analysis indicates measures such as biodata and assessment centers may have d values as large as paper-and-pencil tests of cognitive ability. Personality measures still benefit from small differences. They include a helpful table that breaks down the values by construct, and they also include a list of factors that can impact d, such as job complexity and range restriction.

- Second, a fascinating study of aberrant personality tendencies and their impact on career outcomes conceptualized using the Five-Factor Model and measured using the NEO PI-R. More evidence that "dark side" personality traits are an important consideration in predicting career trajectories.

The other that just came out is the February 2013 Journal of Applied Social Psychology:

- First, a look at moderators of the relationship between employee weight and job-related outcomes.

- Second, a study that I think has implications for selection: looking at circumstances under which competitors copy their opponents choices. I've observed over and over again that when an employee gets a competing offer, suddenly their attractiveness increases. Perhaps not the same phenomenon, but worth exploring.

- Next, another study of discrimination, this time age discrimination in within- and between-career job changes. Results indicated discrimination against older workers was particularly pronounced when older applicants were making between- rather than within-career changes.

- Okay, I'm sensing a theme to this post. In this study, the authors looked at how the wording of occupational descriptions activates gender stereotypes.

- Finally, something not about discrimination: the authors of this (small N) study found that a perceived aspect of emotional intelligence predicted perceived negotiation success beyond traditional personality traits.

And on that note...until next time!

Research update

2013-02-09T21:06:00.000-08:00

It's that time, February research update--and there are some really fascinating studies out there:

First the March 2013 issue of IJSA:

- Procedural fairness is an important aspect of organizational attraction, especially for rejected applicants.

- Does personality type impact perceptions of the selection process? According to this study and their measures (Bohemian is a personality type?), sometimes.

- Do you use SEM to generate a SED and ultimately score bands? Then you will definitely want to read this piece, which suggests the traditional method is flawed.

- Emotional intelligence continues to be a hot topic. In this study the authors describe the development of a SJT-based measure.

- Staying on the topic of personality, this study adds to the debate over the utility of constructs vs. facets in predicting performance--this time with conscientiousness predicting police officer performance.

- P-O fit is often used to explain why people apply for certain jobs. In this study researchers found that values that applicants find attractive are particularly important (as opposed to neutral or aversive aspects).

- I've written about automated reference check systems before. In fact I've written about the company that the first author of this study is from (SkillSurvey). What is partially a demonstration of the value of their product also adds to our knowledge of reference checks by finding no demographic differences and the ability to predict involuntary (but not voluntary) turnover.

- How big of a deal is it when it is discovered that an applicant "embellished" on their resume? Depends on how much you already liked them.

- In this study of applicants in Iran, the authors found that the usefulness of a web-based application system was more important than how easy it was to use. So basically looks are important but not sufficient.

Next, the January issue of the Journal of Applied Psychology:

- Construct validity has always been a sticky wicket. The authors of this study take an interesting approach, using video-based testing to illustrate a method for supporting construct validity.

- This study explores the variance in assessment center ratings stemming from various sources, and the authors argue that existing literature has masked this variance.

-The authors of this study show how the application of a particular bootstrap method applied to meta-analyses increases accuracy.

- Speaking of meta-analysis, the authors of this study demonstrate that the differences between coefficient alpha and composite reliability in actual data sets is quite small.

On to the January issue of the Journal of Applied Social Psychology, which as of the time of this post was available for free!

- Similar to findings in other areas, the impact of negative publicity is stronger than positive publicity on organizational attraction, according to the authors of this study. The authors also found in these cases, organizations would do well to provide detailed recruiting ads to mitigate the impact.

- Age discrimination seems to be increasing in popularity as a research topic (which makes sense from a demographic perspective). In this study, the authors found raters discriminated against older and younger applicants, with the former being the least likely to be hired.

- Here's an interesting one: in this study, people of color perceived White women to be, among other things, ditsy, shallow, privileged, and appearance focused. Stereotypes know no color boundaries.

Let's take a look at the February 2013 issue of the Journal of Personality and Social Psychology:

- First, a fascinating look at how generalizable the five-factor model of personality is across cultures. Specifically, the authors looked at a largely illiterate, indigenous culture in Bolivia. They failed to find support for the widely used model, and suggest that the structure of personality may differ across societies.

- Second, a study of sex and gender differences that points out the importance of considering the construct of gender (as opposed to the binary concept of sex) for a variety of behavioral and psychological indicators, including big five personality.

Let's move now to the February 2013 issue of Psychological Science, where:

- In this study, the authors found that age differences in cognitive functioning within cohorts were often as large as those between cohorts, which calls into question the cohort explanation for age differences.

- In a study that I think has implications for personnel assessment, the authors found that raters of M.B.A. applications that had already given several high recommendations were less likely to do so later in the day. Another bias to watch out for!

Last but not least, there was an article recently in the New York Times about how important referrals are for some organizations and the increased importance this places on social networking sites. There was a reference to a study done in August of 2012 on source of hire that I hadn't seen before, which found that not only were referrals more likely to be hired, they had longer tenure--which is consistent with other research I've seen. Unfortunately, no word about performance differences.

That's all for now!

Of Wal-Mart, veterans, and the nature of jobs

2013-01-19T09:01:00.001-08:00

I think many of us were somewhat impressed with Wal-Mart's announcement that beginning on Memorial Day, they will offer a job to any honorably discharged veteran that applies within a year of ending active duty.

There's no question that the unemployment rate for U.S. vets is too high--consistently 20-30% above the rate for non-vets.

And it's refreshing to see the private sector step up after years of veteran hiring initiatives coming primarily from the government (which is not the driver of employment, at least not recently).

So kudos to them, it continues their public commitment to hiring veterans, and it's a bold gesture (although I wish the one-year time limitation was relaxed).

But what I found even more interesting than the announcement was the reaction from the veteran support community. For example, Yana Walton from the Retail Action Project said (from the article linked above), "Workers are going to make sure that the kind of jobs that they’ve announced for veterans are the kind of jobs that workers need. That means enough hours and living wages and jobs with benefits."

Other workers' rights groups have pointed out that these jobs are likely to include low wages and limited benefits.

Also interesting were the competing stories for why Wal-Mart likes veterans. Wal-Mart's U.S. CEO, Bill Simon, said veterans have a record of performing under pressure, are quick leaders, and team players. It's that last claim that is reflected in a response from Dr. Nelson Lichtenstein, a labor historian at the University of California, Santa Barbara, who said, "'They like military people because they have a sense of hierarchy and a commitment to the organization they are in. And that's important to Wal-Mart.''

Frankly I find both comments to be slightly insulting to veterans, treating them as if they are a homogenous group. I also find it insulting for anyone to assume that veterans are going to leap to embrace any job, regardless of the duties, pay, and benefits.

Some may see this as looking a gift horse in the mouth. I look at it as supporting the dignity of workers and those seeking employment.

One of the most interesting debates that has occurred over the last several years of high unemployment has been about the nature of jobs that people want. Certainly having a job and an income is better than nothing (at least in most cases), but many unemployed aren't interested in any job, they want a job that holds some promise--of good working conditions, of stability, and of promise of future opportunities.

Sadly, that may not be the types of jobs that are being created, at least not in the retail sector. And it begs a larger conversation about education and skill mismatch, as well as how organizations define qualifications.

I also find this fascinating from an assessment standpoint. To promise jobs to a single group of individuals without regard for assessing whether the individual applicants possess the qualifications the employer is seeking for a particular position is, well, frankly stupid from an organizational effectiveness standpoint. This is either a leap of faith on the part of Wal-Mart, or (more likely) a calculated risk given their high turnover rate and the types of jobs veterans are likely to get.

It was with these thoughts rolling around in my brain that I watched my 5-year old's teacher open the door to her kindergarten students the other day. I wondered what it was like for her day after day--the ups (children learning), the downs (noise)--and what attracted and kept her in the job. I would guess that most teachers that stay teachers do so because they receive a significant amount of satisfaction from seeing students achieve and change in positive ways. It's not an easy job, and they don't make a ton of money, so something must be keeping them there.

This made me think about why people apply for jobs at all. Sure, for some people a job is a job--it almost doesn't matter what it is. But I think this is probably the minority of job seekers, and it may be decreasing in prevalence. And the historically (and I would argue misleading) high rate of job satisfaction may be losing its strength as an argument as people ask better questions like whether you would make the same career decision.

So if more and more people are looking not only for a job, but a meaningful, rewarding one, what can we as assessment professionals do?

For starters, we can do more thinking about how to accurately describe jobs. Regular readers know I'm a big fan of realistic job previews and that I think organizations generally do a terrible job at letting applicants know what they're getting themselves into.

Second, we can do a better job at assessing people's motivations for applying. We've all seen the "why are you interested in this job?" question that seems entirely predictable and pointless but often yields surprisingly honest results. If we proceed on the assumption that job performance and retention is driven at least in part by the match between a person's interests and motivations and what the job offers, we do a pretty sad job of measuring this. Most of our measures deal with job-related knowledge or skills. Don't get me wrong--this is really important. But (in addition to personality) it leaves out a huge factor, namely the motivation for applying.

We should be providing more information, asking more questions, digging deeper, investigating new ways of using technology, and treating applicant motivation with a renewed sense of seriousness.

There may have been a time when measuring motivation to apply was considered secondary or unimportant, but that time has passed.

Research update

2012-12-01T16:09:00.000-08:00

Cold/wet yet? Well sit back, have some hot tea, and let's catch up on our research...

Let's start with the biggie: the December issue of IJSA.

- Juggling selection quality and adverse impact continues to be tricky. The authors in this article suggest an optimum combination.

- Reflecting results from the turnover literature, this study found a relationship between perceptions of promotion practices and organizational justice and job satisfaction.

- Speaking of justice, the authors of this study found that the relationship between perceptions of distributive justice and intentions to recommend an employer were moderated by applicant affect.

- Job seekers would to well to keep up their psychological well being and self-esteem (easier said than done, right?).

- Back to fairness. It's reasonable to think that ethnic minority applicants may not perceive video resumes well (due to the increased salience of their minority status). But at least in this study, that assumption was not supported--although it depended on ethnic identity and language proficiency.

- Honestly, I'm not one for the pure concept of "multi-tasking": in my experience people perform in serial, not parallel. But that doesn't stop people from researching the predictive validity of multi-tasking assessments.

- Faking of personality inventories has been one of the hottest topics in assessment for years, but is often framed as dichotomies. This study looks to bring some needed complexity to the issue using qualitative data.

- We all know it can be challenging to get hiring manager to give up their unstructured interviews of questionable validity. Interestingly, according to this study, the more a hiring manager has used unstructured interviews, the less open they are to change. I don't know if this is a causality issue, a mediating variable issue, or more evidence of the inability to accurately judge one's abilities.

- Need more evidence for discrimination that occurs during resume screening? Here ya go.

- Understanding why certain individuals perform better during interviews is a needed area for study. In this article, the authors demonstrate the importance of simply being ability to generate ideas, rather than analyzing the situational requirements.

- This study investigates another area needing more attention: the selection into medical training and education programs. The authors found strong predictive support for SJTs but they came with lower face validity.

- Last in this issue is another interesting study, this time of 360-ratings of innovation. Interestingly and unlike a lot of other research on self-perception, self-ratings were lower than overall observer ratings. However, the situation got more complex when the authors separated and analyzed by level of self-rating.

Next the November issue of Journal of Applied Social Psychology:

- One of the most pervasive (yet bizarrely under-discussed) areas of discrimination in the workplace is age discrimination. This study illustrates some of the stereotypes held of younger and older workers--by both groups. (Spoiler alert: you'll find out how accurate some of these are in just a second)

- I've been waiting for this one, partly because I love hearing about how bizarre and non-face valid some puzzle-based interviews are. This particular study was looking at perceptions of these interviews compared to a behavioral interview. Results? The puzzle-based interviews were consistently less popular. Oh, did I mention that they didn't work as well?

How about the Winter issue of Personnel Psych?

- Why do certain applicants withdraw from the recruitment process? This study suggests a relationship with organizational identification.

- Okay, back to stereotypes about older workers. Whereas the earlier study looked at what the stereotypes are, this one looks at whether they're true. The answer: no, but for one: older workers are less willing to participate in training and career development.

- Turns out it's not just the unemployed that are frustrated by the job search process--currently employed individuals feel the same in many ways. Boy it's too bad we don't have a giant shared database that is able to match job demands with worker abilities...wait...

The November issue of Journal of Applied Psychology has a couple gems:

- Evidence for the predictive validity of the external manifestations of personality as well as the associated implicit motives.

- A reminder that what makes for effective leadership behavior depends on the culture.

The latest issue of Personnel Review has an interesting research article on utility analysis, where the authors reiterate how challenging it can be to communicate UA information (hint: carrot and stick approach may work best).

There are a couple good ones in the November issue of Psychological Science.

- Multiple-choice tests have been beat up in the past for being nothing more than tests of recognition (rather than productive retrieval). This study presents evidence that refutes that assumption. Go multiple-choice!

- Conspiracy theorists take note: governments may be less likely to use the assumed relationship between genetic testing and intelligence to pigeon-hole us into tracks. Why? Because, at least according to this study, there appears to be little evidence connecting the two.

Okay, this one is pretty cool--in a slightly scary way. The authors were looking at the impact that virtual avatar attractiveness has on interview ratings. Turns out our bias toward attractive people is so strong it extends to the virtual world! Of course maybe I should have seen that coming...I mean, ever read a comic book? (hat tip)

Still with me? Last but not least, some disturbing new evidence regarding significance testing and potential publication bias (hat tip). I'm guessing most of you won't be surprised at the finding.

I don't know if I'll have another update before the end of the year, so if I don't, happy holidays to everyone!

Lessons from the SF Giants, 2012 World Series Champions

2012-10-29T15:34:00.001-07:00

I don't use a whole lot of sports metaphors. Heck, I like sports (especially baseball and soccer), but I find they're overused when it comes to HR, especially in team building. Most teams in organizations don't have a defined individual competitor like sports teams do, and their metrics are much harder to pin down.

But in thinking about the dominating performance of the San Francisco Giants during the post-season, culminating in their 4-0 World Series shellacking of Detroit (sorry, Tigers fans), several themes emerged which I think are illustrative. This post won't be about research and it may not immediately seem like it's about recruitment or assessment, but we can extrapolate back from these points to see the implications for selection.

First, workforce planning: it's not always the case (and I know ardent baseball fans will come up with examples) but in the majority of games, certain positions--pitchers--are more important than the others. That's not to say that you can win games without getting runs, but you can come darn close. It's an important reminder that certain positions in an organization/team simply have more leverage and deserve more attention. Bruce Bochy, the Giants' manager, recognized that, leading us to...

Point two: bench strength. It's important to have broad skills across the team, but particularly in your key positions. It speaks volumes about the depth of talent Bochy and the other coaches developed on the Giants that they lost their best closer (Wilson) early on and their former star (Lincecum) never hit his stride as a starter (but more on that in a second), yet they had several talented pitchers they could rely on to step up in these roles. Imagine the flexibility this gives a manager, not only to replace unexpected vacancies but the hand you get to choose from on any given day.

Third: flexibility. Aside from the aforementioned unexpected replacements (not to mention one of their sluggers being lost to a positive drug test), Bochy had to be ready to use people in different ways. When he recognized that a certain catcher worked better with a certain pitcher, he moved the normal catcher (and batting champion, Posey) to first base. When their former ace (Lincecum) couldn't perform as a starter, Bochy found a new role for him as reliever.

Fourth: trust. Bochy kept at least one particular individual (Pence) in the lineup even though he wasn't consistent. Why? Because of his impact on the other players. Bochy was willing to give him some time, and his patience was rewarded with some clutch playing later on--as well as a key motivational impact on the rest of the team, which leads us to...

Point five: fluid leadership roles. Bochy wasn't the only one giving pre-game motivational speeches. In fact the speech most credited with turning the Giants around was delivered by the aforementioned Hunter Pence. Players revved each other up and helped own their team spirit rather than being told to have it. A good manager allows this to happen, doesn't micro-manage, and doesn't have their ego bruised. In fact good leaders will tell you they're happy when they blend into the background of a successful team.

Last but not least: patience. For baseball teams, like all teams, there are rarely simple, quick solutions--this is especially the case in complex organizations with overlapping layers of management, politics, shifting priorities, etc. Sustained success takes years of consistent management with a clear vision. It's been said, but bear's repeating: hiring the best is not good enough. Repeat that five times. It's one of the reasons why assessments don't perfectly correlate with performance, and why a narrow view of selection won't cut it.

So, back to staffing. What are the take home lessons?

1) Certain positions are more important to fill consistently right than others. Does your organization know which ones these are? Is your organization spending its resources acccordingly?

2) Bench strength isn't just important for sports teams. Can your organization withstand the loss of a few key players (no pun intended)? How many people can step up when needed and how quickly will you run out of talent?

3) How flexible is your organization when it comes to putting people in the right place? Are you focused on position statements or on team and organizational success?

4) How much time do you give someone to start performing, and how quick is your organization to judge someone poorly? This has implications not only for things like utility analysis but for organizational culture (and in turn, recruitment...it's a big cycle).

5) When it comes to judging your recruitment and assessment efforts, how integrated is this view with a broader perspective on team and organizational culture? Your organization may be broken into silos but that doesn't mean your perspective has to be.

I promise: limited sports metaphors in the future. But when the cleat fits...

Hiring right: It's about relationships, not technology

2012-10-14T14:04:00.001-07:00

"I'm sorry, Dave. I'm afraid I can't do that." - HAL 9000, 2001: A Space Odyssey

Remember the Wall Street Journal article I told you I was interviewed for? Yeah, well turns out I wasn't quoted, probably because I was a big wet blanket on the topic. But that doesn't matter--what does matter is continuing the conversation that has been sparked by the piece.

The WSJ article talks about using analytics and computers to hone in on what makes for good hiring decisions. In particular, using computers to develop models for the best hiring algorithm. Nothing wrong with that (well, conceptually)--I/O psychologists have been doing that for a long time. I just thought it was sorta old news. But other writers have taken interesting approaches, talking about the implications of "big data" and, well, "big computers", on hiring.

Charles Handler makes some great points in his article on this (including the importance of human judgment and process), as does Joe Murphy in his (e.g., a focus on validation), but I'm going to take a slightly different approach.

Good hiring doesn't hinge on technology. Or validity (bear with me). Or job analysis.

In the practical world, at least in mid- and large-size organizations, good hiring is about the quality of relationship between the HR consultant/recruiter and the hiring supervisor. Because without it, you're guaranteed to introduce bias and lower your success rate.

Why? Because most supervisors aren't experts in hiring. They are, hopefully, experts in their business. But left to their own devices, they WILL ask stupid questions like what animal do you wish you could be. They will make assumptions about things like short job tenure (bad) and attending the same alma mater (good). And they will hamstring the organization from being as effective as it can be as a result of middling talent decisions. But ya know what? I don't blame them--I don't expect them to be good at hiring. That's where HR comes in. HR consultants, like good mechanics, are tasked with diagnosing the situation and coming up with effective, efficient solutions.

The power--and if necessary, blame--lies with the HR function. It controls the relationship and should be held accountable for it. Not sure how well an organization is doing? Here are a few ways of diagnosing how well this relationship is working:

1) How well do the hiring consultants know the business they support? (Yes, this comes first)

2) How well do the consultants know hiring research and best practices?

3) How much are the consultants trusted by their customers?

4) How often do supervisors contact HR about new issues, simply to get their advice?

5) How much unsolicited positive feedback is given to the consultants?

None of this is new. So why am I talking about it? Because it needs to be emphasized. Again and again and again. I've discovered that every time something "new" is discovered in hiring technology, I have to remind folks to go back to the basics.

Technology, or more accurately, good analysis, is essential to ensuring the best ROI on hiring. And as much as I like technology, as much as I have wished sometimes that certain supervisors weren't involved in hiring, they are, and that's the world we live in. And that's not going to change any time soon. Why? Primarily control and arrogance.

First, control. In some organizations, the culture is such that big data may be given priority status over human decision making. But I'm guessing in most it won't (particularly those with merit systems). Because supervisors will not relinquish control over hiring--at least the final decision, and not without a lot of resistance.

Go ahead, take this exercise: think about hiring someone to report directly to you on a daily basis. If I told you I had the perfect algorithm that predicts performance, that you don't even need to meet the person before they start...would you agree to hire them, sight unseen? If you would, kudos: you're unlike 99% of the people I've asked that question.

On to the second part: arrogance. Even after years and years of research with large data sets indicating otherwise, most hiring supervisors believe they know how to judge people and make good hiring decisions. They don't. One thing the focus on big data and the I/O research have in common is a recognition that the more "objective", consistent, and standardized (i.e., less influenced by human biases and decision limitations) you make the decisions, the better they tend to be.

Put these two factors together and you get the status quo, at least for now. Rapidly evolving organizations are more likely to adapt computer-assisted decision making. But in the end, we're unlikely to remove human decision-makers from controlling final hiring decisions in many organizations. For now.

But perhaps that's not a bad thing. As bad as our biases can be, the errors that entire computer systems can cause are much more serious. If only I could think of an overblown example...

"The system goes on-line August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th." - Terminator 2, Judgment Day

Research update: September, 2012

2012-09-30T07:57:00.000-07:00

It's been a while since I've posted a research update. I was waiting for a critical mass but honestly it was taking a while! But no more waiting, let's see what's out there...

First up, a topic near and dear to my heart: HR certification. In the September/October issue of HRM, Lyons, et al. took a look at a sample of web-based HR job ads to study the prevalence of requiring the PHR/SPHR certifications offered by SHRM, er, the the Human Resource Certification Institute. Back in 2005, a similar study found that less than 2 percent of jobs listed the certifications as preferred or required. This time? 15.6 percent. Boom! Quite an increase (although obviously still far from the majority). Now the important question: does possession of said certificates actually predict job performance? I gotta be totally honest, I've seen some blind admiration of these certifications without any indication that this question was addressed.

Anyhoo, what else is in that issue of HRM? For one, a study of perceived supervisor support and team-level performance. Subjects: 75 gas stations in Norway (I just wanted to say that). Results? Link between the two. Implication? Another competency to consider when hiring supervisors. Oh, and btw, there's another study in the same issue about HR practices in the organization and links to OCB and customer service (hint: participation is good).

Like soccer (or, should I say, football)? How about team research? Either way you'll be interested in this fascinating study, which found that players transferred to another team improved the performance of their new team against their old one.

Finally, this small study which found having applicants complete self-affirming written self-guidance statements prior to interviewing improved their performance.

Okay, next up is the September issue of JAP.

How about another leadership study? Why not. In this one, researchers found supervisor consideration behaviors were positively related to employee attitudes (and apparently the more the merrier) while there was an ideal level of initiating structure behavior. There's a lot more with this one, particularly on P-E fit, so check it out.

Have you ever found yourself wondering, "I wonder what the power of cross-level interaction effects is when conducting tests of multilevel contingency and interactionism"? Well today's your lucky day, because check this out.

Promoting and fostering diversity in organizations is hypothesized to have various positive outcomes, one of which is creativity. But previous results are mixed. In this lab study, the authors propose an explanation: it depends on the extent to which team members take each others' perspectives. Another reminder that simply recruiting and hiring people with different backgrounds does not ensure successful performance.

Anyone performing differential item functioning (DIF) analysis should check out this study, which recommends a particular procedure for identifying optimal anchor items.

Finally, and this doesn't directly relate to assessment but is just darn interesting, a study on "cyberloafing" and the impact that moving to daylight savings time has on it (hint: it's not good).

Okay, back to the topic at hand and the October JOB.

Need another example of the complexity of the relationship between personality factors and organizational behavior? Check this out (personality factor: conscientiousness).

Speaking of conscientiousness, this study found that adaptive performance (acquiring new competencies as a result of organizational change) was related to task performance but the relationship was impacted by employee conscientiousness (and organizational politics!).

Did someone say conscientiousness? No, seriously, another study on it. But it's another interesting one that looks at the importance of group perceptions as well as team composition when analyzing the relationship between individual personality measures and performance. And it's another sports team study, this time university football players (U.S. football that is).

Last but not least, the October issue of JPSP. And the first study may just be the most interesting thing I've seen in a while.

In it, the authors demonstrate via lab and field experiments with a variety of subjects and decisions (including graduate school admission), that people seem to have a preference for potential rather than achievement. What are the implications for recruiting and assessment? That decision makers may be unduly influenced (at the cost of validity) by the promise of an applicant rather than their demonstrated accomplishments. However, this warning must be moderated with an acknowledgement that sometimes potential (e.g., ability scores) can be equally valid sources of predictive validity. So, bottom line: another bias to watch out for.

Next, an elucidating if slightly depressing study of stereotype threat. The authors demonstrate that the experience of stereotype threat among African Americans and Hispanic/Latino(a)s resulted in scientific disidentification and intention to pursue a scientific career.

Okay, this next one is tricky because it looks interesting but I wasn't able to find an in-press version (maybe a helpful reader can point one out?). It's a refinement of a theory of basic individual values, and looks like it has implications for career and applicant selection.

Finally, very last but most definitely not least, a study of the accuracy of personality judgments, a particularly timely topic given recent research that suggests these judgments have significant value in personnel assessment. Specifically, the authors were looking at hindsight effects (essentially how your perception changes after time and additional information). Good stuff and implications for anyone wanting to use observer measures of personality.

Is "big data" relevant for recruitment and selection?

2012-09-01T13:40:00.000-07:00

Recently I was contacted by a reporter from the Wall Street Journal about the application of the "big data" movement to predictive analytics and HR topics like recruitment and assessment. I wanted to share a little about my reaction and my thoughts afterward.

For the unitiated, the big data movement is all about stocking, mining, and extracting information from large datasets. In the modern HR world, this data is typically found in human resources management systems--systems that contain information like time to fill, assessment results, compensation, and performance management measures.

Why has this become a hot topic? It's hard to say. Could be the buzz generated by Moneyball, the book and movie about the Oakland A's GM Billy Beane and his use of data to uncover data points (e.g., on base percentage) that bucked existing predictive measures such as batting average. Or it could be Google's study of what predicts supervisory success (shocker: it's not technical skill). I suspect it's also been fueled not insignificantly by software and consulting companies hoping to capitalize on the interest.

When I was chatting with the reporter, I found it difficult to answer some of the questions about the application of this movement to recruitment and assessment. It was only later that I realized why: for us, this is an old idea.

Asking if the analysis of large data sets is applicable to selection is like asking if sunshine is applicable to farming: it's a foundation upon which the practice exists. Not only is the research that surrounds our field grounded in data analysis, the major impactful discoveries have been in large part based on the analysis of large data sets -- things like the predictive power of cognitive ability and conscientiousness, and the U.S. Army's Project A. The entire profession of personnel psychology is founded upon the idea that through analyzing data we can help answer big questions like what predicts job success, leadership, and organizational attraction.

So I'm guessing I'm not the only one who is observing this trend and thinking: what took you so long?

Now, does all this mean the big data movement is pointless? Absolutely not. To the extent that organizations are renewing their interest in using data analysis to guide decisions, booyah. But here are my big four concerns:

1) The results are only as good as the data. Let's say you sick your analytical software on your HR data and find out that candidates recruited from the Northeast don't perform as well as those recruited from the South. Easy enough, looks like we need to shift our recruiting resources. But not so fast. What is your performance measure? What if I told you that the majority of your supervisors come from the South--might that impact the results? What if one of the success criteria is knowledge of customers in the South--but you're interested in expanding into other territories? Without first looking deeply at what our measures are, we risk coming to some very misleading conclusions. (Some of you will recognize this as "the criterion problem").

2) There's analysis, and then there's analysis. Anybody can run a correlation. But do you know about power? Statistical significance versus practical significance? Multivariate analysis? Collinearity? If that sounds like gibberish, please seek professional help.

3) With apologies to Kurt Lewin, nothing is as practical as a good theory. What if you find out that answers to "what's your favorite color" predict success as a senior manager? What does this mean? We can make all kinds of guesses, but without having a theoretical framework in place, we're letting results drive "the truth" rather than logically positing a relationship and seeing if the data support it. Basically what I'm saying is: beware fishing expeditions. All you have is a correlation; don't make the mistake of inferring causation.

4) What do you do with the results? So employees that drive to work outperform those that take public transit. Does that mean you force all your employees to drive? What if your analysis uncovers something uncomfortable about your current executive leaders--what the heck do you with that? (a: bury it, b: post and pray, c: use consultantspeak to obfuscate results, d: re-run analysis).

Without first thinking about these and other important questions, the "big data" movement, when applied to important questions like what predicts organizational behavior, has the potential to create all kinds of erroneous, wasteful, and potentially risky conclusions.

On the other hand, if this ends up creating an additional sense of energy around evidence-based management, I may end up looking back at this as an extremely positive development in helping organizations succeed.

Research update

2012-08-07T12:06:00.000-07:00

Okay, way past update time. Let's take a look at the latest research (this month's themes: core self evaluations, SJTs, and, of course, personality tests):

Let's start with a couple from the July issue of Journal of Applied Psychology:

- First, an application of signaling theory to selection. The authors point out that viewing selection through this lens, where the focus is on the honesty of communication between applicant and employer, can help shed light on the field and point to future directions.

- Speaking of honesty, the other study is about a proposed way to reduce faking on personality tests. Specifically, the authors looked at the efficacy of providing applicants feedback about their honesty midway through the test; looks like they found mixed results.

Next, let's look at one from the August issue of the Journal of Organizational Behavior:

- The authors studied the impact that psychological capital (e.g., optimism, self-efficacy) has on job search behavior. They found a positive relationship between capital and perceived employability, which itself was related to various good (i.e., problem-focused) and not-so-good (i.e., symptom-focused) coping strategies.

Next, two from the Autumn Personnel Psychology:

- More support for the idea of contextualizing personality inventories. What does that mean? Essentially tailoring the test for work situations, and better yet specific work environments. In this study the mean criterion-related validity jumped from .11 (non-contextualized) to .25 (contextualized).

- Second, what looks like a fascinating study of what factors impact applicant attraction at various stages in the recruitment process. Interestingly, perceived fit was the strongest predictor of attraction but was not a significant predictor of job choice (the strongest predictor was job characteristics). In addition, organizational characteristics and recruitment process characteristics became more important in later stages.

Okay, those were warm-ups. Let's get into the heavy hitter, the September issue of the International Journal of Selection and Assessment:

- First, more on the importance of core self evaluation (CSE). In this study the authors found support for CSE explaining incremental variance in performance over ability and conscientiousness. They propose that CSE does so through its impact on learning motivation.

- Think situational judgment tests can't be coached? Think again.

- Should O*NET information be based on analyst ratings or incumbent ratings? Yes. Looks like each provides value.

- Applicant reactions: always a popular topic. This time the location is Mumbai, India. Not surprisingly, resumes and interviews fared well, while graphology and honesty tests did not. However, in an interesting twist work sample tests were rated unfavorably.

- Do recruiters care about volunteer experience? Not really.

- Might test-takers get fatigued at the end of a long SJT? Yes. Might it impact the psychometric properties? Yes. Might it impact subgroup differences? Umm...sort of.

- How many different ways can you analyze the reliability of an SJT? Turns out, quite a few.

- Aaahhhh yes, emotional intelligence. Haven't heard from you in a while. In this study the authors found positive applicant reactions and incremental validity over ability and the Big 5 across three samples (for the MSCEIT).

- Last but not least, let's end with another personality test--well, integrity test really--the venerable Personnel Reaction Blank. This time the authors looked at cross-cultural generalizability (U.S. and Singapore) as well as gender differences.

That's all for now!

The hidden world of promotions

2012-07-04T16:58:00.001-07:00

For whatever reasons, there are a few topics related to recruitment and assessment that just don't get the attention they deserve. The competency levels of the HR community is one of them, but today I'd like to focus on another: promotions.

Despite the incredible importance the promotion process plays in organizations, it receives scant attention. It impacts morale, teams, organizational leadership, motivation, engagement, creativity...the list goes on and on. Yet most of the energy in our area seems to be focused on seeking out new talent. So let's look at some factors related to promotions that reinforce why this topic deserves our attention.

When hiring for entry-level jobs, everyone in the organization has the same objective: get the best person (i.e., most qualified) on board--barring side factors like people wanting to hire family members. Supervisors want the highest performer, as do most co-workers, who have no personal career stake in the selection. I say most because there is the occasional employee who, for fear of being outshined, could conceivably want to hire a poorer performer to keep their reputation up. But typically they aren't the decision makers.

Promotions, on the other hand, involve a whole host of other factors, which, again, I think deserve more attention:

1) Internal competition. The sheer fact that individuals within the organization are competing for this position--often against teammates--changes things. It means there may be harsh feelings within a team, and at the very least it means employees in different areas may get competitive. Now competition isn't necessarily a bad thing--sometimes it helps people raise their performance to new levels. But don't underestimate the long-term impact of this mostly silent phenomenon.

2) Losers. Because this is a competition, not everyone is going to get the position. How will you deal with the aftermath of those not chosen remaining in your organization? How do you communicate the decision and how do you turn the conversation into one focused on future opportunities? What do you do about an employee who is denied promotions repeatedly?

3) Fallout. The individuals that aren't selected have friends and connections. You know that old shampoo commercial where the happy customer told two friends, and they told two friends, and so on and so on? Same here, except the message may not be positive.

4) Factors other than observable competencies. Unfortunately we know that factors unrelated to strict job-related competencies enter into the assessment process. Because promotions involve current employees, the potential for this happening is increased. Sometimes this means someone's reputation (established through, say, gossip) hinders--or helps--their chances. Other times it's more political, with selection choices being made to turn in favors (or for the promise of future ones).

5) Past job performance. Because these individuals already work for the organization, typically there is a trove of information about them in their current (and maybe past) roles. This information is undoubtedly important, but how do you combine it with assessment scores? Equally important, is performance in a previous role necessarily directly related to performance in a future one? A great secretary doth not necessarily a great analyst make (neither should he/she be dismissed).

6) Acting assignments. During transition periods and before the recruitment is completed, employees are often asked to take on acting roles. This provides employees with a valuable opportunity to gain experience at a level above their own. But what happens when the person who is acting in the role doesn't get the job? On the flip side, is there an assumption that the person on the acting assignment will automatically get the job?

7) New roles. Often individuals promote from a staff position to a position supervising their former team. This can be uncomfortable for everyone involved, and if the supervisor doesn't make adjustments to his/her relationships, it will make it difficult to administer discipline, etc.

8) The domino effect. Filling positions with internal talent almost always results in another vacancy. How easy or difficult will it be to fill that position? Is it fair to deny an opportunity to someone based on how valuable they are in their current role?

So what are the implications of all this?

1) Post-selection communication is key. Think upfront about how you will communicate the choice, the decision factors, and opportunities moving forward. Don't underestimate the grapevine that exists in the organization and how your every move will be interpreted and communicated. And be honest with those that don't get the job about what they need to improve upon to be more competitive.

2) All high potentials should have career opportunities. The promotion process should not be the only opportunity to have this discussion, it should be a regular topic.

3) Reward people for stepping in when needed for acting assignments (assuming they do a good job). Some temporary assignments offer much in the way of responsibilities but not so much in the way of additional compensation. However, that person gains valuable experience and gives decision makers a chance to observe what they might be like in that position. This should be given consideration, otherwise you won't see very many hands go up the next time you ask for volunteers.

4) If you're promoting to a first-line supervisory position, provide these individuals with significant support. In some ways individuals promoting into supervisory roles need more support than those just joining the organization. The challenges are more complex, as are the relationships. Technical skill no longer cuts it and the abilities to coach, motivate, and hold people accountable become key. These aren't skills most people are born with, so make sure you get them started right.

5) Have a holistic assessment strategy. I'm likely preaching to the choir, but know ahead of time how you will combine scores on different assessments and how you will factor in knowledge of current job performance. Do not make the mistake of ignoring what you know about the person, but think critically about transferability of skills.

I'm sure I just skimmed the top of this iceberg, but my goal here is to encourage discussion of this important topic. Any manager that has been faced with a promotion decision knows (or should know) how complicated they are. Your employees--and your organization--will benefit from some deep thinking about the issue.

Hurry and register for the 2012 IPAC conference!

2012-06-26T19:09:00.002-07:00

You only have two more days to take advantage of the "early bird" registration rate for the 2012 International Personnel Assessment Council (IPAC) conference in Las Vegas on July 22-25.

This year's conference promises to uphold the high quality of presentations and opportunities to network that you've come to expect. Great keynote speakers are lined up, such as Wayne Cascio, Kevin Murphy, and David Campbell.

Check out the flyer and the preliminary program to see all the goodness!

I hate resumes

2012-06-09T20:55:00.000-07:00

Okay, I know last time I talked about how I hate interviews--or at least how they're used. I promise this blog isn't turning into 100% rants. But some things ya just gotta get out.

My dislike for interviews is far eclipsed by my distaste for resumes. In fact the process of reviewing resumes as a selection device is far worse than most interviews.

Why? Because, like interviews, resume requirements are ubiquitous, but research has shown their validity (i.e., the relationship between resume scoring future job performance) is worse.

Aside from straight up not working very well as an assessment tool, what else is wrong with resumes? Here's a sample:

- They're not standardized. You'll get some information on some resumes, different information on others. So you end up comparing apples and oranges. Not exactly what you hope for in terms of reliability.

- They don't lend themselves easily to a scoring rubric. Sure, you can try giving a certain number of points for a degree or a certain amount of experience (assuming you can support it), but that's assuming the person writes it down in the way you need...Otherwise, it's easy to let them fall into "piles" (definitely yes, maybe, hell no). Unfortunately at that point you're essentially saying there is no variance within each pile, which is obviously not true.

- They're open to all kinds of bias, ranging from racial discrimination to wild assumptions about education ("hey, look--they went to the same school I did! cool!")

- They're time consuming to review. We've all seen resumes the size of small novels (hint to job seekers: this does not impress most employers). Separating the wheat from the chaff is about as much fun as a 4:30 meeting.

- Like training and experience (T&E) measures, they tell you nothing about how well someone did something, just that they did it. Assuming they're telling the truth, which leads to...

- You're forced to take on faith that the education and experience they describe was actually obtained. Sure, you'll verify it (right?), but at the very least you've wasted time including liars/embellishers in the process.

- The information you need may not be there. You will almost certainly find yourself thinking, "gosh, I wish they had given me more detail about X..."

- On the other hand, they contain a lot of information you don't need. Applicants try to put their best foot forward and are also guessing what you're looking for, so they waste their time and yours. They're just plain inefficient--for applicants and employers.

So why do employers use them? For a lot of the same reasons they recycle interviews:

- It's easy; heck, just put "resume required" on the ad...because...well...why not? It doesn't require us to think very deeply about the recruitment.

- It's the devil we know. It's what people are comfortable with. People may acknowledge their imperfection, but this personality testing stuff sounds scary...

- Inertia. Yes, unfortunately "it's what we've always done" has an inordinate amount of influence on the way people in organizations make decisions.

- A lack of attention to and focus on the process. The reason many selection processes are lackluster is because the hiring supervisors don't take it seriously enough. Who's fault is this? Well theirs, obviously, but HR's as well, and upper management. Hiring should be one of the most important things employees do.

- They don't know any better. This is perhaps the saddest reason because it's simply due to a lack of education about selection. Many hiring supervisors are unaware of the range of assessments out there, or maybe they've heard vague rumors about how they can be sued if they use an intelligence test. HR and upper management, I'm looking at you again on this one.

- They're not held accountable for the quality of their hires and the selection process they're using. Heck if requiring resumes is easy, it seems to mostly work, and no one's the wiser, why would I change my ways?

So aside from educating supervisors and holding them accountable, what can be done? There have been several web-based attempts to make resumes more relevant. But so far there is no "killer app" that both job hunters and employers make a beeline for. There's no Facebook of resumes (and no, Facebook is not the Facebook of resumes). In the meantime, here are some tips:

- Don't default to asking for a resume. Really think about what you're looking for. Interested in their job-related education, training, and experience? Then ask for that! Like...

..."Specifically describe how your experience and training matches the requirements of the position..."

...Or even better: "This job requires an advanced ability to put peanut butter on bread. Describe specifically how your training and experience qualifies you to perform this task."...

- Don't assume that standard applications solve the problem. They tend to have a lot of the same problems--e.g., the candidate gets total flexibility in describing their previous jobs.

- If you need, for whatever reason, to ask for a resume, try asking for it in a specific (chronological, functional) format. And be clear about what it should--and should not--contain. And limit the length. Who knows, it may at least speed things up for you.

Resumes and interviews. Probably the most frequently used personnel selection tools used. And sadly in many cases severely lacking in validity. Really the problem gets us back to the core of good selection: taking hiring seriously and spending time thinking about job requirements and a plan for assessing them.

These steps aren't that hard, and simply taking a more critical look at them may give you more payoff than any other revamp of your hiring and promotion process.

I hate interviews

2012-06-05T10:03:00.001-07:00

I hate interviews.

Let me clarify. I don't hate doing interviews, I hate how they're used.

Try this exercise. Spread your arms out as far as they'll go (or envision it). That's how much of a person's competencies employers think they are measuring using the interview.

Now put your hands about a foot apart from each other. That's how much of a person's competencies they're really measuring. And that's if they interview well (which many don't).

So why do employers like interviews so much? Yes, when done right they've been shown to predict performance pretty well, so it makes sense that over time supervisors would see them as a decent way to hire the right person (although how many of them correlate interview scores with later job performance?).

But I think it's more basic than that. Interviews are the basket that many employers put all their eggs. Why? It's been argued that they're addicted to interviews and stubbornly refuse to acknowledge their flaws. But really I think there are three main things going on with the average hiring supervisor:

1) competing demands for their time/attention;

2) laziness ("let's just do what we did last time"); and

3) a genuine belief that SEEING someone tells you a lot about them. I believe this factor is particularly powerful.

Here's another exercise: ask a bunch of hiring supervisors how many of them would hire someone without interviewing them. I'm guessing no one volunteered.

Why? Why is it so uncomfortable to imagine hiring someone without having seen them? What are they hoping to measure in the interview?

- Job knowledge? Heck, we could put together a 10-15 item m-c test that would do a better job than a couple "what would you do if..." and "tell me about a time when..." questions. The slight additional time involved in putting them together would be worth the increased confidence in the breadth/depth of their knowledge.

- Oral communication? First of all, is that important for the job? Second, is an interview really the highest fidelity situation you can think of? Are you hiring for...interviewee? Ask them to put on a training presentation. Do a role play. Be creative!

- "Something else"? Like, I dunno, friendliness? Social skills? "Fit"? Do you really think you're seeing an accurate picture of these qualities during an interview? Why not use an assessment that's actually designed to measure these things and reference check the heck out of the person?

Aside from the variance you're picking up on job knowledge and communication, really the best you can hope for is that the person messes up. Pity the poor applicant who hasn't memorized the organization's webpage and job bulletin, smiles continuously, and knows which magic words to utter. Instead, what if they:

- Show up to the interview late
- Wear something weird/inappropriate
- Complain inordinately about previous coworkers/bosses
- Ask you bizarre questions

I don't have any data to back this up, but I'd guess nine times out of ten when it comes down to the final group of applicants the "there was just something about them" factor trumps. But can it be quantified? Defended?

So what can be done? There are two big leverage points that organizations need to focus on to avoid the automatic recycling of interviews:

1) HR consultants need to be assertive and available. They should be contacting supervisors when they know about a vacancy, when the advertisement goes out--WHENEVER--just make contact. Find out how you can help. As a hiring supervisor myself, I can tell you it makes a HUGE difference when a consultant asks me if there's anything I can do (to which I respond, "God, yes!").

2) Supervisors--and HR--need to be held accountable for their hires and their reputation as a destination employer. Supervisors, if you have the resources available and you simply fail to take advantage of them, if you never look into why you're having recruiting problems but just keep using the same methods, shame on you. HR, if you know you should be providing this service and you aren't, or if you know you should be better at it but aren't, shame on you.

So it's a big problem, but it's one that with sufficient attention can be tackled. If it seems like I'm being overly negative about interviews and their real-world application, all I can say is...you should see how I feel about resume reviews.

May mega research update

2012-05-26T12:10:00.000-07:00

It's time once again for the monthly research round-up. So let's dive right in:

The June International Journal of Selection and Assessment doesn't disappoint; let's take a look:

- More evidence of the link between personality variables and CWBs; this time with concurrent data in China

- Dovetailing nicely with a post I've been working on regarding promotional testing, this research indicates some interesting characteristics of internal test takers

- Why are supervisors open to behavioral interviews but shun discussion of "structure"? Looks like how we communicate about them plays a big role.

- More research on self-efficacy, this time teasing apart the concept a bit.

- Always a popular topic: applicant reactions to selection mechanisms. This time with a sample from Saudi Arabia.

- Speaking of applicant reactions...how about another study? This one comparing U.S. and Vietnamese college students. By the way, not surprisingly work samples came out a winner in both of these studies.

- Next, a fascinating study of a hidden bonus to UIT: despite the cheating element, it likely increases your candidate pool and eventually performance outcomes

- Speaking of response distortion, here's another study, this time of military cadet selection using personality inventories

- Okay, one more on inflation. This time a study of Chinese applicants--no difference compared to American samples.

- Back in March I wrote about a study Jeremy Bernerth published in J.A.P. that got a lot of attention. This time, Bernerth studied ethnic differences and found minority status was negatively related to credit scores.

Moving on to the summer issue of Personnel Psychology:

- The "file drawer problem" is the theory that nonsignificant results are less likely to get published. According to this study, that appears unlikely. But IMHO looking at all correlations is different than looking at the correlations key to one's hypothesis(es)...

- Back to faking (that may be this post's theme!), can response elaboration reduce faking on biodata items? This study suggests so. Although I'm left wondering...what was the impact on validity?

- Speaking of biodata, there are various ways of keying these items. This research suggests the best method depends on your sample size, although rational keying performed the worst.

How about the May issue of Journal of Applied Psychology?

- Well this is interesting...Chad Van Iddekinge and his colleagues have provided an updated meta-analysis on the criterion-related validity of integrity tests. What did they find? Well, the results appear to be less promising than those published previously (e.g., corrected r=.18 for job performance). Much like SIOP's research journal, this time J.A.P. published several commentaries in response to the study that...well, let's just say a debate ensued about the analysis...

- The Dark Triad. It sounds like something in a Dan Brown novel. But in this meta-analysis the authors show that personality characteristics that make up this triad (Machiavellianism, narcissism, and psychopathy) explain some variance in CWBs.

- Why are some people more proactive in seeking career goals than others? It's an important and under-researched question. In this study the authors show that part of the explanation lies in "future work selves", or how people's hopes and aspirations as they relate to work.

- Think self-reports of CWBs are biased? Perhaps not, according to this new study.

- Interested in what causes proactive customer service behavior? According to this multi-national study, self-efficacy is a key (along with service climate).

- Why do some leaders engage in more self-interested behavior than others? Perhaps not surprisingly, it appears due in part to the strength of their moral identity.

The May issue of the Journal of Applied Social Psychology has a couple gems...

- Hey, look, turns out being sensitive to your subordinates pays off. Talk about a lesson that needs frequent repeating...

- And that's it. Oh, wait, just this little study about using Facebook profiles to predict job performance...that I wrote about before....available in FULL right now...

Okay, getting to the end...The May/June issue of HRM:

- An interesting study of adverse impact in promotion decisions for managers in a Fortune 500 retailer. The authors compared three methods (top-down assessment, assessment centers, and multisource appraisal) and the results demonstrate how complex these analyses are!

- Speaking of complex. Think that successful job postings on the web is just fancy graphics? Think again--it still involves some classic factors like the labor market, firm reputation, and compensation incentives. The more things change...

- Identifying future leaders. There are few other issues that are as important for most organizations. Yet how exactly to do it eludes many. These authors propose a model that focuses on four main features: analytical ability, learning agility, drive, and emergent leadership.

Finally, a few from PARE:

- Does item order impact response anxiety? Not according to this study.

- What's that? How do we use a new jacknife procedure to eliminating items and improve structural equation modeling? You're in luck.

- Looks like a lot of research rely on beta weights when interpreting and reporting multiple linear regression results. But there's so much more...

Happy reading!

What The Avengers can teach us about high-performance teams

2012-05-06T14:07:00.000-07:00

Like millions of other people, I saw Marvel's The Avengers yesterday. It's anticipated to bring in around $200M in its opening weekend. It's full of explosions, space aliens, and people in outlandish costumes.

So what does The Avengers have to do with recruiting and hiring? Nothing. Unless you care about building a high-performance team.

Sure, the movie is nominally about a egomaniacal godlike being, Loki, who is attempting to destroy all humans using an army of aliens summoned through a space portal (I call that Monday). But I think what the movie's really about is the challenge of building and sustaining a high-performance team.

The power of teamwork

Nick Fury has a tough job. As the Director of a secret international espionage and military agency, he's tasked with figuring out how to respond to the threat that Loki poses. While his bosses (let's call them the board) encourage him to use more drastic solutions, Fury steadfastly sticks to his team (that'd be The Avengers--Captain America, Iron Man, Black Widow, Thor, Hawkeye, and The Hulk). Why? Because he believes that their sum exceeds their parts.

Which brings us to our first lessons:

1) Teams are particularly effective when you need the benefits that come with combining talents and skills to deal with a complex situation beyond the abilities of any individual.

2) Successful team leaders believe passionately in the power of the team and are personally devoted to seeing them succeed. They may even need to resist pressure from above and put their reputation on the line.

Dealing with Superstars(heroes)

Fury's biggest challenge lies in assembling the team and keeping them cohesive. Why? Because each one of them is a "high producer". Technically they don't need each other to do great things--and each one of them is extremely confident in their abilities (with Iron Man, played by Robert Downey, Jr. being the poster boy). Each is used to dealing with big challenges themselves, in their own way. Not only are they not used to a "boss" (Fury), but they struggle to form a group identity. Lessons:

3) Recognize that building a team of superstars will be a challenge. And then recognize it publicly. Acknowledge to the team that accomplishing great things will likely not be easy--but it is doable (and in this case, necessary).

4) Plan ahead for what will attract these individual performers to come together. Is it broader recognition? Satisfaction of a job well done? A sense of duty? An opportunity to right wrongs?

5) Prepare for some in-fighting. This is likely inevitable among individuals used to working in their own fashion (and, not coincidentally, getting all the glory).

Bringing them together

So how do you recruit and keep together a group of often-selfish, always stubborn, personalities? Fury gives us some clues...

6) Seek out the diverse talents you need. While each member of The Avengers is super in their own right, each also brings something different. For example, Black Widow is particularly adept at persuasion and interrogation; Captain America is a natural leader; and Thor, well...Thor happens to be related to the main bad guy so he's pretty familiar with the core issues.

7) Appeal to a greater cause. Fury isn't shy about sharing his passion for the idea of The Avengers: the world's most powerful superheroes coming together to defeat evil that threatens the planet. That's a pretty powerful EVP and/or leadership vision, wouldn't you say? (By the way, he's also good at manipulating team emotions for the greater cause)

8) Harness the unique talents of your team members. While Fury is a pretty good recruiter in his own right, he recognizes that certain team members (namely Dr. Bruce Banner a.k.a. The Hulk) might be better wooed by others (i.e., Black Widow). Similarly, as the two scientists, Iron Man and Bruce Banner, are brought together we see immediate results of their complimentary passion and talent.

Keeping them together

Here's where Fury stumbles a little, and it results in the biggest setback the team experiences. The team starts bickering and Fury lets himself get drawn into it. This intensifies the mistrust and distracts them while their enemies infiltrate their headquarters. Only the immediate threat solidifies the team. Last lessons:

9) As a leader, stay above the fray. Teams with strong personalities don't need another one. Your job is to stay and produce calm. Fury would have been better served by calling for a time out so people could cool their jets.

10) Stay focused. Don't let distractions such as momentary setbacks or petty infighting ruin the potential. Remind people why they're there.

11) Be honest. In Fury's case, he's caught with hidden intentions and it drastically lowers team trust. Superstars are often particularly adept at spotting weaknesses in leaders. Don't give them reason to doubt.

There's a lot of research and writing on the topic of building and sustaining high-performance teams. Heck, there are even conferences devoted to it. Interestingly, it's one of the most enduring themes in graphic novels as well (think Fantastic Four, X-Men, etc.). There's a reason why there is so much interest: there are times when special teams are called for, and it's exciting to think about harnessing disparate talents and focusing them on achieving great things.

The best lessons, I think, to draw from The Avengers are that bringing together superstars isn't easy, and keeping them together may be even more difficult. And it's another example of where the strength of leadership can make or break the mission. We may not be trying to recruit and engage superheroes. But we should all be familiar with the challenges inherent in bringing individuals together in the pursuit of a common goal. Even if it isn't saving the world.

A fascinating example of an organization making hiring job #1

2012-04-21T20:31:00.001-07:00

Recently the employee handbook of Valve, a software and video game development and distribution company (famous for things like Half-Life and it's Steam service) was leaked and frankly it's one of the most interesting things I've read in a long time. It's available several places, including here as of this writing.

Why so interesting? Several reasons...

1) It doesn't look or read like a typical employee handbook. It's very informal, devoid of legalese, balances positivity and expectations, and is graphically very attractive. Not exactly signs of your typical handbook. As an example, the document starts with this statement on the cover:

A fearless adventure in knowing what to do when no one’s there telling you what to do.

(They do keep their formal policies on their Intranet (e.g., about benefits).)

2) It's a very interesting example of how a flat organization describes itself. They don't appear to have much of a management hierarchy so much of the handbook is devoted to explaining how employees are supposed to select assignments, etc.

3) The document itself is editable on their Intranet. Yes, you read that right, the employee handbook is a collaborative document.

Last but definitely not least,

4) This is a great example of how an organization can emphasize that hiring is the most important activity employees engage in. The document is replete with examples. Consider the following passages from the handbook:

p. 6: "hiring is the single most important thing you will ever do at Valve"

p. 14: "We have made significant strides toward bringing more predictability, measurement, and analysis to recruiting. A process that many assume must be treated only as a “soft” art because it has to do with humans, personalities, language, and nuance, actually has ample room for a healthy dose of science."

p. 17: "The thing we work hardest at is hiring good people"

here's my favorite:

p. 44: "Hiring well is the most important thing in the universe. Nothing else comes close. It’s more important than breathing. So when you’re working on hiring—participating in an interview loop or innovating in the general area of recruiting—everything else you could be doing is stupid and should be ignored!"

p. 45: "Missing out on hiring that great person is likely the most expensive kind of mistake we can make...a poor hiring decision can cause lots of damage, and can sometimes go unchecked for too long."

They also talk about how they hire:

p. 47: "When unchecked, people have a tendency to hire others who are lower-powered than themselves...We should hire people more capable than ourselves, not less."

pp. 47-48: "[In some circumstances] hiring someone who is at least capable seems (in the short term) to be smarter than not hiring anyone at all. But that’s actually a huge mistake. We can always bring on temporary/contract help to get us through tough spots, but we should never lower the hiring bar."

There are so many examples it's almost more of a introduction to hiring rather than to the company!

Also, in case you're interested, here's their "work at valve" page, which supports the culture as described in the handbook.

This "leak" has gotten a lot of press, and likely has done a lot to increase its attractiveness as an employer--another reason why this is such a fascinating example.

So I ask you: how much of your organization's handbook discusses the importance of hiring? Do they make it clear it's a shared responsibility of every employee? Is there a passion for hiring right?

April research update

2012-04-01T18:08:00.004-07:00

Okay, so I didn't quite hit my March update...date. But I'm awful close, so without further ado let's take a look at what research has come out lately. And boy is there a lot to cover...

First, the Spring Personnel Psychology, which it should be noted is all free at the time of this writing (!):

- Moore, et al. describe the development and test of a measure of "an individual's propensity to morally disengage", which is really (as the title suggests) about investigating why people do bad things at work. Looks like it has promise beyond existing measures that predict things like CWBs.

- Next, O'Boyle and Aguinis present the results of a fascinating study of the distribution of human performance. Turns out it may not be normal after all, but rather Paretian. This has big implications for...well, all sorts of things, including selection. Read here for more.

- Avery, et al. describe results of a study of racioethnic matching (employees and customers) and the impact on productivity (which turned out to be positive, through customer satisfaction). The authors present several very practical ways of interpreting this finding without jumping to hiring based on race.

Next, the March Journal of Applied Psychology:

- First, van Hooft and Born with a fascinating study of eye-tracking to investigating faking on personality and integrity measures. Looks like eye movement differs depending on the intent to inflate, and it also suggests response time could be an indicator of inflation.

- Next, Madera and Hebl with another eye-tracking study, but this time on the impact that facial stigmatization has on interview performance. Discouragingly (but perhaps not surprising), the results suggest individuals with facial stigmatization may receive lower ratings, in part due to the interviewer being distracted.

- Into core self-evaluations? You might want to read this study by Wu and Griffin, in which they argue that CSEs are predictors of, but also influenced by, contextual factors such as job satisfaction.

- Lievens and Sackett provide evidence that individuals' procedural knowledge of interpersonal behavior may be valuable in predicting performance (in this case, medical students).

- Bernerth and colleagues discuss the usefulness of credit scores in predicting job performance, which I wrote about in an earlier blog post.

Next up, the May Journal of Organizational Behavior:

- Peng, et al. suggest that deployed soldiers with higher levels of conscientiousness and lower levels of neuroticism may be able to better cope with psychological distress (the effect of extraversion was mixed).

- Derous, et al. discuss discrimination in resume screening among individuals who belong to multiple minority groups (in this case with a focus on Arab women). Applicant, job, and recruiter characteristics were all important.

The March Industrial and Organizational Psychology has a fascinating focal article on how I/O psychology and HR can contribute to organizational strategy and competitive advantage. Several commentaries follow, at least one of which directly addresses selection. Move quick, because right now both the focal article and the commentaries are free!

Now here are some miscellaneous articles you may be interested in:

- Jackson, et al. describe a study that suggests going through military training may alter someone's personality...

- Using a 1- or 2-item measure of personality traits, thinking you don't need more than that? Think again.

- Looking for creative thinkers? Gino and Ariely provide evidence that may give you pause: those that are the most creative may be more dishonest...(!)

- Hiring people into a particularly political environment (naw, none of us ever have that)? Chang, et al. present results that suggest you may want to pay attention to their self-monitoring skill and level of conscientiousness...but maybe not in the way you think.

- Still getting over St. Patrick's Day? Then check out van den Born and van Witteloostuijn's research on "shamrock" organizations. They suggest this type of organization may explain the conflicting findings on modern job tenure.

Finally, all the presentations from the 2011 IPAC conference that were previously available only to members have been made public. There is so much good stuff here I can't even begin to summarize it. Just go check it out.

Facebook fallout continues

2012-03-28T18:41:00.004-07:00

The fallout from earlier reports of employers asking applicants for their Facebook passwords continues. Obviously a nerve was struck.

Today, U.S. Senators Richard Blumenthal and Charles E. Schumer formally asked the Department of Justice (DOJ) and Equal Employment Opportunity Commission (EEOC) to launch an investigation into whether this practice violates federal laws.

From the press release:

"Blumenthal and Schumer argued that this disturbing practice represents a grave intrusion into personal privacy that could set a dangerous precedent for personal privacy and online privacy, make it more difficult for Americans to get jobs, and expose employers to discrimination claims"

"'With few exceptions, employers do not have the need or the right to demand access to applicants’ private, password-protected information.'”

"'In an age where more and more of our personal information – and our private social interactions – are online, it is vital that all individuals be allowed to determine for themselves what personal information they want to make public and protect personal information from their would-be employers. This is especially important during the job-seeking process, when all the power is on one side of the fence.'"

"In their letter to the Justice Department, Blumenthal and Schumer pointed out that two courts have found that when supervisors request employee login credentials, and access otherwise private information with those credentials, that those supervisors may be subject to civil liability. Although those two cases involved current employees, the courts’ reasoning does not clearly distinguish between employees and applicants."

"Blumenthal and Schumer also announced that they are currently drafting legislation that would seek to fill any gaps in federal law that allow employers to require personal login information from prospective employees to be considered for a job."

In related news, on today's Talk of the Nation show, they discussed this issue with a reporter from Wired magazine and an HR consultant.

There are a lot of issues here, ranging from online privacy to public reputation to discrimination, but one that I think deserves more attention is how employers can legitimately get the type of information they're seeking. Again, we're not talking about a background check for, say, a peace officer position, we're talking about your run-of-the-mill clerical job. Basically employers are hungry for any information like displays of poor judgment, a negative attitude about their employer, duplicity in their application, etc.

How might an employer get this type of information without resorting to asking for applicant passwords? It's pretty simple actually, we go back to the basics such as:

1) Reference checks; highly under-used and maligned, with many organizations unaware of technological advances made in this area that make it more likely they'll get the information they need.

2) Work sample/performance tests that simulate actual job tasks. These can be very effective in determining how an applicant will respond in an actual situation (i.e., where things like judgment are important).

3) Situational judgment tests: a lower fidelity version of a performance test that nonetheless can be very effective at assessing candidate's knowledge of, and propensity to engage in, appropriate behavior in various situations.

4) Personality inventories: made to measure things like conscientiousness, openness to experience, and extraversion, which may all be good or bad things depending on the needs of the position.

Bottom line: there are other--better--forms of assessment out there that have been around for a long time and when done well, do the trick. No need to ask for someone's online diary.

As a reminder, for those of you that are IPAC members, I gave a webinar about this topic about a year and a half ago where I gave an overview of the technology as well as a summary of many of the major challenges inherent in this practice. The recording is available in the Members Only area.

Epic fail: Employers continue asking applicants for Facebook account

2012-03-23T18:45:00.008-07:00

The Associated Press put out an article this week about the ongoing trend of employers doing something stupid: asking applicants for their Facebook passwords.

Why stupid? Let me count the ways:

1) It makes the applicants feel like they've applied to a totalitarian regime. And they'll tell others, which goes to your reputation. And what do we know about employer reputation? It drives who applies for your jobs.

2) Employers are likely to see things they wish they hadn't. I don't just mean people passed out drunk at a party, I mean things like religious affiliation.

3) If you're trying to access their profile on your own, many are marked private and you won't see anything.

4) If you ask them to log into their account during the interview, it's like asking to see their personal diary.

5) The content on people's FB page is largely outside their control (e.g., comments, photos they're tagged in).

Oh, and let's not forget:

6) The content of someone's profile--aside from things like education and work history which you should have already--is likely to be totally unrelated to job performance, regardless of its potential usefulness, because frankly most employers aren't graduate students in psychology who have received training on interpreting Big 5 characteristics.

The only caveat I can think of is when this request is made as part of a full background check, in which case pretty much your life is an open book.

Facebook, notoriously unpredictable regarding its privacy policies, subsequently warned employers not to do this...but I don't anticipate that this will stop. Why? Because employers are obsessed (rightfully so) with getting as much--and as varied--information as they possibly can.

This just isn't the right way to do it.

Has your organization given up on rigorous assessment?

2012-03-16T06:36:00.003-07:00

I'd like to hear from you:

Has your organization given up on rigorous assessment? Given in to the incessant demands for "faster, cheaper"? Sold its soul to the T&E gods? Failed to replace seasoned, trained, and passionate assessment experts and replaced them with generalists?

Or have you gone the opposite direction--are you innovating and experimenting with new forms of assessment (like the feds are)? Are you selective in whom you choose to work on assessment? Do you resist efforts by management to "dumb down" your selection processes?

Credit scores: Useful for selection, but not for the reason you think

2012-03-14T06:35:00.004-07:00

Many employers use credit scores as part of their hiring process, despite the lack of evidence regarding their usefulness, the unpopularity of this practice with applicants, and the fact that the EEOC is not a fan.

Adding to our understanding of the issue, in the March 2012 issue of Journal of Applied Psychology (which I will review fully in my March research update), Jeremy Bernerth and his colleagues describe the results of their study where they found:

1) A significant positive relationship between credit scores and task performance

2) A significant positive relationship between credit scores and OCBs

most likely due to

3) Credit scores being positively related to conscientiousness

because there was

4) No significant relationship between credit scores and workplace deviance, such as theft

Interestingly, credit scores were negatively correlated with agreeableness, which the authors say suggests that more agreeable people are more likely to do things like co-sign on questionable loans.

So the bottom line is credit scores may be valuable because they link to performance ratings and OCBs (likely through personality), not because they predict things like theft.

Which leaves the obvious question: why not just use a personality inventory, which is designed to measure conscientiousness and has little adverse impact (unlike credit scores)?

February 2012 research update: Facebook; hope; 3-option items; cognitive ability and racism

2012-02-26T07:28:00.015-08:00

So this month the big buzz is over a research article in JASP on using Facebook profiles to judge personality and predict job performance. So let's tackle that one first:

Kluemper, et al. had three trained university students judge Big Five personality factors (using the IPIP) based on over 200 public Facebook profiles of other students. A much smaller sample (56) was used to determine links between evaluator judgments and job performance as rated by supervisors.

So what did they find? Well, among other things:

(1) inter-rater reliability ranged from .48 to .72 which seems low to me but apparently is typical for other-ratings of personality;

(2) two of the other-ratings (emotional stability and agreeableness) significantly correlated with supervisory ratings (around .30), as did one of the self-ratings (extraversion), suggesting to me the two methods might vary in constructs being measured;

(3) the same two other-ratings added incremental validity beyond self-ratings, whereas the opposite was not true.

To their credit, the authors caution readers (those that actually go beyond the mainstream press articles) about using their results to support hiring decisions based on Facebook. In fact, I'd like to quote them:

"Our findings should not be used by organizations as unbridled support for using SNWs [Social Networking Websites] in employment selection. Without more evidence of criterion-related validity and comparability with established employment selection methods, the use of SNW information for hiring purposes is tenuous. In addition to the potential for employment discrimination, there are privacy rights and ethical issues associated with accessing personal information. Clearly, research investigations of such issues lag current informal HR practices."

So bravo to them for researching this issue, and double-bravo (that's a technical term) for cautioning those with Facebook fever.

Let's move to other research out there...

The March issue of IJSA is chalk-full of great research, so let's take a look:

- Much hay is made over the "type" of validity exhibited by cognitive ability tests (don't get me started). To the extent this distinction makes sense to you, you might enjoy Frank Schmidt's argument that ability tests in certain instances can demonstrate content validity. The article is followed by several commentaries and a response.

- Next up is O'Neill, et al. with a critical review of Stevens and Campion's Teamwork-Knowledge, Skills, and Ability Test.

- Reeder, et al. investigate individual differences as they relate to the perception of cognitive ability tests.

- Now here's somethin': Edwards, et al. argue that the three-option multiple choice item is underutilized. This should be read by everyone who has nightmares about writing distractors.

- Hoffman and Meade argue that score differences across assessment center exercises reflect true differences rather than measurement artifact.

- Got hope? Zysberg shows that, through problem-solving-oriented coping, hope is related to success in a selection process.

We already know that people generally aren't very good at accurately describing their skills and abilities, but in the March Psychological Bulletin, Freund and Kasten provide an illuminating meta-analysis indicating that while the relationship between self-estimated and psychometrically measured cognitive ability is modest (.33), it varies depending upon scales and dimensions.

For anyone measuring contextual performance, consider the relationship between role expectations and OCBs as described by Dierdorff, et al.

Speaking of OCBs, Nielsen et al.'s study suggests when measuring OCB expression in a group setting, consider the level of task interdependence.

On the other hand, if you are interested in job performance ratings, be aware that there may be gender bias, and it differs in direction between performance ratings and promotability ratings (with the latter favoring males), according to a recent meta-analysis by Roth, et al.

Last but definitely not least, research by Hodson and Busseri suggest that individuals lower in cognitive ability may be predisposed to exhibit more racism. This suggests that using ability tests may not only increase the validity of your selection process but lower your chances of discriminatory behavior.

****

On a side/editorial note, I find it fascinating and somewhat frustrating that the research energy still seems to be about teasing out major constructs such as cognitive ability and personality. As a practitioner, I gotta tell ya I'm happy when hiring supervisors use any sort of structured assessment beyond their standard interview. I'd love to see more energy behind increasing the validity of the entire selection process. I doubt I'm the only one that feels that way.