Why Validity Matters

Posted by Robert Hogan on Fri, Dec 03, 2010

Personality psychology concerns three questions. First, in what important ways are people all alike? This question involves analyzing the nature of human nature. Second, in what important and systematic ways are people all different? This question concerns individual differences. The third question concerns how to measure, in a reliable and valid manner, important individual differences in personality? These measures can then be used to predict practical outcomes—e.g., job performance, career and financial success.

But why are personality data so useful? The reason is simple: (1) The best predictor of future behavior is past behavior; (2) A person’s personality (defined in terms of observer’s ratings) is the summary of that person’s past behavior; so that (3) Personality (defined in terms of observers’ ratings) is the best data source we have regarding a person’s future behavior. Our assessments are keyed into observer ratings and provide an objective way to predict a wide variety of life outcomes, including life expectancy, marital satisfaction , substance abuse, and career and financial success.

The principal statistic used in our research (as well as research in economics, sociology, and medicine) is the Pearson correlation coefficient (r). The value of r can range from -1 (a perfect negative association between two variables) to 0 (no relationship between two variables) to +1 (a perfect positive association between two variables). The validity coefficients found in most medical research are below .20. For example, the correlation between smoking and contracting lung cancer within 25 years is .08; the correlation between taking ibuprofen and reduced pain is .14. Typical validities in psychological studies tend to be higher. For example, the correlation between applicants' scores in a personnel selection interview (which is an inefficient form of personality assessment) and subsequent job performance is .30, and the correlation between IQ scores and school grades is .70. Our research over the past 30 years has produced validity coefficients that are significantly higher than those typically found in published medical or economic research.

Personality and industrial/organizational psychologists use correlation coefficients to predict individual differences in peoples’ present or future performance. The best way to interpret a correlation is in terms of hits and misses. Imagine we have tested 200 sales candidates on the HPI Ambition scale, and then hired all of them; Figure 1 shows the expected percentages of high and low performers as a function of the validity of the ambition scale, using validities of .00, .20, .30, and .50.
 


If the validity coefficient for the Ambition scale in predicting sales performance is 0, then 50 of the new hires with above average Ambition scores will be high performers and 50 will be low performers (a validity coefficient of 0 implies that the prediction based on test scores is no better than chance). If the validity coefficient is .20, then 60 of the new hires with high Ambition scores will be high performers and 40 will be low performers, whereas 60 of the new hires with low scores will be low performers and 40 will be high performers. If the validity coefficient is .30, then 65 of the new hires with high scores will be high performers and 35 will be low performers; conversely 65 of the new hires with low scores will be low performers and 35 will be high performers. If the validity coefficient is .50, then 75 of the new hires will be high performers, and 25 will be low performers, etc. Thus, validity coefficients allow us to estimate how much better than chance (50%) we can predict performance (e.g., if r = .20, then we will predict 10% better than chance or we will make the correct prediction in 60% of the cases; if r = .60, then we will predict 30% better than chance or we will make the correct prediction in 80% of the cases).

The importance of this becomes obvious when we consider that the top 25% of employees in any job or profession will contribute 400% more than the bottom 25% of employees. The major point, however is that validity coefficients are the primary index for estimating the practical significance of any test.

Topics: validity coefficients, performance, assessment validity

Subscribe to our Blog

Most Popular Posts

Connect