When discussing the topic of selection assessment with human resources professionals, it can be rather easy to overwhelm a non-technical audience by carrying on about job analysis, criterion validation, correlations, legal defensibility, etc. A former colleague of mine who worked as a sales representative used to say I was getting “I/O-ish” (as in Industrial/Organizational psychology) when I started using such terminology. Keep in mind that I’m the first person to advocate the merits of assessment validation for ensuring effective talent management solutions. However, my colleague made an important point that sometimes, in an effort to provide the details behind the psychometrics of implementing an assessment for candidate selection, we may inadvertently add complexity to the conversation. 

In order to provide a simple structure to explain the process for implementing a selection assessment, I devised what I’ve coined the “which hunt.” That is not a typo. I’m not referring to a witch hunt as in the Salem witch trials of the late 1600s, nor does what I’m proposing resemble the McCarthyism of the 1950s. My concept of a which hunt is a series of discovery questions that an organization must answer to create a solid foundation for a high-quality assessment strategy that will support the identification of high potential candidates during the pre-employment screening process. An effective which hunt will help a company to identify:
• WHICH characteristics should we measure?
• WHICH assessment(s) should we use?
• WHICH cut-score will increase our hit rate for identifying good candidates?

WHICH characteristics should we measure?
You cannot hope to measure the potential for a candidate to be successful until you define which characteristics lead to success in a specific job. The Occupational Information Network (O*NET) is an online resource sponsored by the Department of Labor that reports profiles for over 800 occupations.  O*NET provides 277 data points for each occupation covering abilities, skills, knowledge, work styles, and other characteristics. Clearly, if O*NET is any indication, the process of defining a job profile of required characteristics can be a daunting proposition to an organization looking to implement an assessment program to measure such characteristics.
 
Rather than immediately getting into the details of job analysis or competency modeling, I find that a simple description of “can-do aptitudes” and “will-do attitudes” helps communicate the likely outcomes of this first step in the which hunt process. Can-do aptitudes refer to the mental horsepower of candidates such as cognitive abilities, demonstrated capability for job-specific skills, and mastery of specific areas of job knowledge. Put simply, having these aptitudes indicates that you can do the job, but we all know that not everyone lives up to their potential.

The will-do attitudes are often those characteristics that allow employees to meet their potential and can even lead an employee with less raw ability to actually succeed beyond those seemingly more talented colleagues. These work styles include conscientiousness, interpersonal savvy, stress tolerance, and achievement orientation, among others. Very smart, very talented employees often fall short of their full potential or fail because they do not work hard, do not play well with others (customers and/or teammates), and do not effectively manage pressures at work. 

Taken all together the required can-do and will-do characteristics form the success profile for that specific job.

WHICH assessment should I use?
The next step is to identify an assessment that measures these characteristics in candidates. As we do with most ventures in life, we might begin the search for an assessment provider by using our good friend Google. When you type “candidate selection assessment” into Google, the result includes over 1.5 million hits! Which one should you choose?

My message to human resource professionals is that the best assessment is one that measures the critical components of the success profile you identified in the first step of the which hunt. Your assessment strategy doesn’t have to measure the entire success profile (that’s why we conduct interviews, administer basic qualification questions, collect resumes, etc.), but there should be significant overlap. This may require the implementation of multiple assessments. For example at Hogan, we offer the Hogan Business Reasoning Inventory to measure can-do characteristics, the Hogan Personality Inventory and Hogan Development Survey to measure will-do characteristics, and the Motives, Values, Preferences Inventory to assess “will-fit” characteristics in terms of how a candidate might fit into the organization’s culture.

WHICH cut-score will increase our hit rate for identifying good candidates?
Now for the last leg of the which hunt – interpreting the assessment results. While this task seems easy enough – low scores fail and high scores pass –   the truth is that this can actually be quite challenging for multiple reasons. What do you do with scores in the middle? Is a 40 a good enough score or should I look for 50s and higher? If I have multiple scores for multiple characteristics, how do I know what scores are more important indicators of success? What if the candidate has high scores on some characteristics and low scores on others? It is a delicate process to draw the line in the sand and make appropriate sense of all the good information that assessments provide. 

This part of the which hunt gets a bit complicated despite my best efforts. The bottom line is that we must clearly understand the relationship between assessment scores and job performance. I find it helpful during this part of the conversation to use an analogy for the way a financial institution uses a credit score. If a bank is going to give someone a loan to purchase a house, they don’t just want to get a high level summary of age, income, salary, credit card debt, etc., and shoot from the hip on how to combine all of those data points into an estimation of investment risk for that person. Such an approach would be inconsistent, inaccurate, and not scalable. To make sound lending decisions over time, the bank leverages a proven, weighted equation to combine these data points into an easily interpretable credit score that is backed by research to increase the hit-rate for making profitable lending decisions (the recent housing market collapse aside). Standards have been set to categorize bad credit scores, good credit scores, and great credit scores. This is essentially how a validation study is used when implementing an assessment for candidate selection. We conduct research to give you overall low, moderate, or high evaluations of candidate potential that if used consistently will increase hit rates for selecting successful employees.

The which hunt guidelines break up the concepts of assessment implementation into concepts that are hopefully simple to grasp:  
• Before we can measure anything we must define a benchmark (i.e., success profile).
• We must use that benchmark to guide us to pick the right tool for the job (i.e., assessment).
• We have to know how to read the measurements the tool is giving us and do regular checkups to make sure the measurements are accurate (i.e., cut-scores).

When discussing the topic of selection assessment with human resources professionals, it can be rather easy to overwhelm a non-technical audience by carrying on about job analysis, criterion validation, correlations, legal defensibility, etc. A former colleague of mine who worked as a sales representative used to say I was getting “I/O-ish” (as in Industrial/Organizational psychology) when I started using such terminology. Keep in mind that I’m the first person to advocate the merits of assessment validation for ensuring effective talent management solutions. However, my colleague made an important point that sometimes, in an effort to provide the details behind the psychometrics of implementing an assessment for candidate selection, we may inadvertently add complexity to the conversation. 

In order to provide a simple structure to explain the process for implementing a selection assessment, I devised what I’ve coined the “which hunt.” That is not a typo. I’m not referring to a witch hunt as in the Salem witch trials of the late 1600s, nor does what I’m proposing resemble the McCarthyism of the 1950s. My concept of a which hunt is a series of discovery questions that an organization must answer to create a solid foundation for a high-quality assessment strategy that will support the identification of high potential candidates during the pre-employment screening process. An effective which hunt will help a company to identify:
• WHICH characteristics should we measure?
• WHICH assessment(s) should we use?
• WHICH cut-score will increase our hit rate for identifying good candidates?

WHICH characteristics should we measure?
You cannot hope to measure the potential for a candidate to be successful until you define which characteristics lead to success in a specific job. The Occupational Information Network (O*NET) is an online resource sponsored by the Department of Labor that reports profiles for over 800 occupations.  O*NET provides 277 data points for each occupation covering abilities, skills, knowledge, work styles, and other characteristics. Clearly, if O*NET is any indication, the process of defining a job profile of required characteristics can be a daunting proposition to an organization looking to implement an assessment program to measure such characteristics.
 
Rather than immediately getting into the details of job analysis or competency modeling, I find that a simple description of “can-do aptitudes” and “will-do attitudes” helps communicate the likely outcomes of this first step in the which hunt process. Can-do aptitudes refer to the mental horsepower of candidates such as cognitive abilities, demonstrated capability for job-specific skills, and mastery of specific areas of job knowledge. Put simply, having these aptitudes indicates that you can do the job, but we all know that not everyone lives up to their potential.

The will-do attitudes are often those characteristics that allow employees to meet their potential and can even lead an employee with less raw ability to actually succeed beyond those seemingly more talented colleagues. These work styles include conscientiousness, interpersonal savvy, stress tolerance, and achievement orientation, among others. Very smart, very talented employees often fall short of their full potential or fail because they do not work hard, do not play well with others (customers and/or teammates), and do not effectively manage pressures at work. 

Taken all together the required can-do and will-do characteristics form the success profile for that specific job.

WHICH assessment should I use?
The next step is to identify an assessment that measures these characteristics in candidates. As we do with most ventures in life, we might begin the search for an assessment provider by using our good friend Google. When you type “candidate selection assessment” into Google, the result includes over 1.5 million hits! Which one should you choose?

My message to human resource professionals is that the best assessment is one that measures the critical components of the success profile you identified in the first step of the which hunt. Your assessment strategy doesn’t have to measure the entire success profile (that’s why we conduct interviews, administer basic qualification questions, collect resumes, etc.), but there should be significant overlap. This may require the implementation of multiple assessments. For example at Hogan, we offer the Hogan Business Reasoning Inventory to measure can-do characteristics, the Hogan Personality Inventory and Hogan Development Survey to measure will-do characteristics, and the Motives, Values, Preferences Inventory to assess “will-fit” characteristics in terms of how a candidate might fit into the organization’s culture.

WHICH cut-score will increase our hit rate for identifying good candidates?
Now for the last leg of the which hunt – interpreting the assessment results. While this task seems easy enough – low scores fail and high scores pass –   the truth is that this can actually be quite challenging for multiple reasons. What do you do with scores in the middle? Is a 40 a good enough score or should I look for 50s and higher? If I have multiple scores for multiple characteristics, how do I know what scores are more important indicators of success? What if the candidate has high scores on some characteristics and low scores on others? It is a delicate process to draw the line in the sand and make appropriate sense of all the good information that assessments provide. 

This part of the which hunt gets a bit complicated despite my best efforts. The bottom line is that we must clearly understand the relationship between assessment scores and job performance. I find it helpful during this part of the conversation to use an analogy for the way a financial institution uses a credit score. If a bank is going to give someone a loan to purchase a house, they don’t just want to get a high level summary of age, income, salary, credit card debt, etc., and shoot from the hip on how to combine all of those data points into an estimation of investment risk for that person. Such an approach would be inconsistent, inaccurate, and not scalable. To make sound lending decisions over time, the bank leverages a proven, weighted equation to combine these data points into an easily interpretable credit score that is backed by research to increase the hit-rate for making profitable lending decisions (the recent housing market collapse aside). Standards have been set to categorize bad credit scores, good credit scores, and great credit scores. This is essentially how a validation study is used when implementing an assessment for candidate selection. We conduct research to give you overall low, moderate, or high evaluations of candidate potential that if used consistently will increase hit rates for selecting successful employees.

The which hunt guidelines break up the concepts of assessment implementation into concepts that are hopefully simple to grasp:  
• Before we can measure anything we must define a benchmark (i.e., success profile).
• We must use that benchmark to guide us to pick the right tool for the job (i.e., assessment).
• We have to know how to read the measurements the tool is giving us and do regular checkups to make sure the measurements are accurate (i.e., cut-scores).