Our clients often request a benchmark against which to evaluate their employees on our personality assessments. A common statement is “we want to see how our sales force compares to other sales forces.” Some are even more specific, such as “we want to benchmark our managers against other managers working in large, pharmaceutical companies in East Asia.” Although we always want to be accommodating, we tend to shy away from creating such benchmarks for multiple reasons. The most important reason is that, devoid of the relevant contextual information, benchmarks using personality assessment are not very useful and can lead to false conclusions.
The desire for benchmarks in personality assessment is likely a product of normal business thinking. As organizations, we are always comparing ourselves to our competition to know where we stand. Stock price, annual revenue, number of products sold, customer retention rate and satisfaction ratings, safety violations, and employee turnover rates. These are all things that indicate a clear winner when we are benchmarking ourselves against our competitors. Relative to other organizations, we want higher stock value, revenue, sales, and customer ratings. We want lower safety violations and employee turnover.
Being better than your competition is not as clear when it comes to personality assessment. The general bias when interpreting personality assessment scores is that higher scores are better. This bias stems from our orientation to other standardized testing; we know that higher scores are generally better when evaluating ourselves on tests of intelligence, aptitude, skills, and abilities. The reality is that there are strengths and shortcomings associated with any score on any scale from most decent personality assessments. No one gets off the hook. Although people who score high on our Ambition scale have more natural drive and goal-orientation than the population, they may not recognize when it is time to let others lead and may be perceived as poor team players. For example, let’s assume that an oil company has requested a personality benchmark for their safety inspectors relative to other safety inspectors. We create this benchmark, present it to the oil company, and they see that their safety inspectors are higher on a scale measuring creativity than other safety inspectors. Higher is better, right? Not really. Safety inspectors shouldn’t be creative. Instead, good safety inspectors should be more practical, less creative…following the letter of the law. In this case, lower scores on the creativity scale would be better.
Another bias is that client organizations (and people, in general) often assume that any comparisons we, as experts, make must be relevant and attended to. When ESPN reports on the average number of water bottles kept on the sidelines by World Cup teams, we assume that this has some bearing on the success of the team, or else why would they have reported it? Naturally, the googlesphere will activate with fans searching to find out how many water bottles their team keeps on the sidelines. Companies could compare themselves on the number of staplers they have but what does that matter as a metric of business performance? Further, is it better to have more or less staplers? When it comes down to it, not all comparisons are meaningful. As it applies to personality assessment, not all personality characteristics are related to performance (relevant) in a given job or industry. How important for job performance is it for a custodian to be sensitive to the feelings of others? It is likely irrelevant, but we can still create a benchmark that compares the interpersonal sensitivity of one school’s custodians to all other school custodians. When we provide this benchmark, the client organization is likely to assume it has relevance, or else why would we have reported it?
We feed into these biases when we provide benchmarks without the necessary contextual framework. When an organization sees their employees’ personality data plotted against an industry/subgroup benchmark, they may make inaccurate inferences because a) we always assume higher is better and b) we assume that all reported comparisons are important. We can create meaningful benchmarks with personality assessment but it takes the right kind of data. If we have sufficient data, we are able to indicate what personality scales have the strongest and most consistent relationships with performance in these jobs; hence, relevance. In addition, we can indicate whether higher, lower, or even moderate scores are better. Obviously, the thinner we slice the world (e.g., marketing coordinators at mid-western hotel chains), the less likely it is that we, or anyone, will have sufficient evidence to reliably indicate predictors of performance and create good benchmarks. Further, such specific benchmarks lack relevance and applicability. Truth be told, there probably isn’t much difference in personality between marketing coordinators at mid-western hotel chains and marketing coordinators in general. With all stated caveats, we can help clients benchmark their employees on characteristics that matter for performance, as long as it is a worthwhile endeavor and we have the data to do it. Even then, we should not be creating stand-alone graphs or tables that simply plot group averages against each other. We must tell the story around and behind the graph or table to focus attention appropriately. Only then can a personality benchmark be meaningful, impactful, and actionable.
