Our clients often request a benchmark against which to evaluate their employees on our personality assessments. A common statement is “we want to see how our sales force compares to other sales forces.” Some are even more specific, such as “we want to benchmark our managers against other managers working in large, pharmaceutical companies in East Asia.” Although we always want to be accommodating, we tend to shy away from creating such benchmarks for multiple reasons. The most important reason is that, devoid of the relevant contextual information, benchmarks using personality assessment are not very useful and can lead to false conclusions.

The desire for benchmarks in personality assessment is likely a product of normal business thinking. As organizations, we are always comparing ourselves to our competition to know where we stand. Stock price, annual revenue, number of products sold, customer retention rate and satisfaction ratings, safety violations, and employee turnover rates. These are all things that indicate a clear winner when we are benchmarking ourselves against our competitors. Relative to other organizations, we want higher stock value, revenue, sales, and customer ratings. We want lower safety violations and employee turnover.

Being better than your competition is not as clear when it comes to personality assessment. The general bias when interpreting personality assessment scores is that higher scores are better. This bias stems from our orientation to other standardized testing; we know that higher scores are generally better when evaluating ourselves on tests of intelligence, aptitude, skills, and abilities. The reality is that there are strengths and shortcomings associated with any score on any scale from most decent personality assessments. No one gets off the hook. Although people who score high on our Ambition scale have more natural drive and goal-orientation than the population, they may not recognize when it is time to let others lead and may be perceived as poor team players. For example, let’s assume that an oil company has requested a personality benchmark for their safety inspectors relative to other safety inspectors. We create this benchmark, present it to the oil company, and they see that their safety inspectors are higher on a scale measuring creativity than other safety inspectors. Higher is better, right? Not really. Safety inspectors shouldn’t be creative. Instead, good safety inspectors should be more practical, less creative…following the letter of the law. In this case, lower scores on the creativity scale would be better.

Another bias is that client organizations (and people, in general) often assume that any comparisons we, as experts, make must be relevant and attended to. When ESPN reports on the average number of water bottles kept on the sidelines by World Cup teams, we assume that this has some bearing on the success of the team, or else why would they have reported it? Naturally, the googlesphere will activate with fans searching to find out how many water bottles their team keeps on the sidelines. Companies could compare themselves on the number of staplers they have but what does that matter as a metric of business performance? Further, is it better to have more or less staplers? When it comes down to it, not all comparisons are meaningful. As it applies to personality assessment, not all personality characteristics are related to performance (relevant) in a given job or industry. How important for job performance is it for a custodian to be sensitive to the feelings of others? It is likely irrelevant, but we can still create a benchmark that compares the interpersonal sensitivity of one school’s custodians to all other school custodians. When we provide this benchmark, the client organization is likely to assume it has relevance, or else why would we have reported it?

We feed into these biases when we provide benchmarks without the necessary contextual framework. When an organization sees their employees’ personality data plotted against an industry/subgroup benchmark, they may make inaccurate inferences because a) we always assume higher is better and b) we assume that all reported comparisons are important. We can create meaningful benchmarks with personality assessment but it takes the right kind of data. If we have sufficient data, we are able to indicate what personality scales have the strongest and most consistent relationships with performance in these jobs; hence, relevance. In addition, we can indicate whether higher, lower, or even moderate scores are better. Obviously, the thinner we slice the world (e.g., marketing coordinators at mid-western hotel chains), the less likely it is that we, or anyone, will have sufficient evidence to reliably indicate predictors of performance and create good benchmarks. Further, such specific benchmarks lack relevance and applicability. Truth be told, there probably isn’t much difference in personality between marketing coordinators at mid-western hotel chains and marketing coordinators in general. With all stated caveats, we can help clients benchmark their employees on characteristics that matter for performance, as long as it is a worthwhile endeavor and we have the data to do it. Even then, we should not be creating stand-alone graphs or tables that simply plot group averages against each other. We must tell the story around and behind the graph or table to focus attention appropriately. Only then can a personality benchmark be meaningful, impactful, and actionable.

Our clients often request a benchmark against which to evaluate their employees on our personality assessments. A common statement is “we want to see how our sales force compares to other sales forces.” Some are even more specific, such as “we want to benchmark our managers against other managers working in large, pharmaceutical companies in East Asia.” Although we always want to be accommodating, we tend to shy away from creating such benchmarks for multiple reasons. The most important reason is that, devoid of the relevant contextual information, benchmarks using personality assessment are not very useful and can lead to false conclusions.

The desire for benchmarks in personality assessment is likely a product of normal business thinking. As organizations, we are always comparing ourselves to our competition to know where we stand. Stock price, annual revenue, number of products sold, customer retention rate and satisfaction ratings, safety violations, and employee turnover rates. These are all things that indicate a clear winner when we are benchmarking ourselves against our competitors. Relative to other organizations, we want higher stock value, revenue, sales, and customer ratings. We want lower safety violations and employee turnover.

Being better than your competition is not as clear when it comes to personality assessment. The general bias when interpreting personality assessment scores is that higher scores are better. This bias stems from our orientation to other standardized testing; we know that higher scores are generally better when evaluating ourselves on tests of intelligence, aptitude, skills, and abilities. The reality is that there are strengths and shortcomings associated with any score on any scale from most decent personality assessments. No one gets off the hook. Although people who score high on our Ambition scale have more natural drive and goal-orientation than the population, they may not recognize when it is time to let others lead and may be perceived as poor team players. For example, let’s assume that an oil company has requested a personality benchmark for their safety inspectors relative to other safety inspectors. We create this benchmark, present it to the oil company, and they see that their safety inspectors are higher on a scale measuring creativity than other safety inspectors. Higher is better, right? Not really. Safety inspectors shouldn’t be creative. Instead, good safety inspectors should be more practical, less creative…following the letter of the law. In this case, lower scores on the creativity scale would be better.

Another bias is that client organizations (and people, in general) often assume that any comparisons we, as experts, make must be relevant and attended to. When ESPN reports on the average number of water bottles kept on the sidelines by World Cup teams, we assume that this has some bearing on the success of the team, or else why would they have reported it? Naturally, the googlesphere will activate with fans searching to find out how many water bottles their team keeps on the sidelines. Companies could compare themselves on the number of staplers they have but what does that matter as a metric of business performance? Further, is it better to have more or less staplers? When it comes down to it, not all comparisons are meaningful. As it applies to personality assessment, not all personality characteristics are related to performance (relevant) in a given job or industry. How important for job performance is it for a custodian to be sensitive to the feelings of others? It is likely irrelevant, but we can still create a benchmark that compares the interpersonal sensitivity of one school’s custodians to all other school custodians. When we provide this benchmark, the client organization is likely to assume it has relevance, or else why would we have reported it?

We feed into these biases when we provide benchmarks without the necessary contextual framework. When an organization sees their employees’ personality data plotted against an industry/subgroup benchmark, they may make inaccurate inferences because a) we always assume higher is better and b) we assume that all reported comparisons are important. We can create meaningful benchmarks with personality assessment but it takes the right kind of data. If we have sufficient data, we are able to indicate what personality scales have the strongest and most consistent relationships with performance in these jobs; hence, relevance. In addition, we can indicate whether higher, lower, or even moderate scores are better. Obviously, the thinner we slice the world (e.g., marketing coordinators at mid-western hotel chains), the less likely it is that we, or anyone, will have sufficient evidence to reliably indicate predictors of performance and create good benchmarks. Further, such specific benchmarks lack relevance and applicability. Truth be told, there probably isn’t much difference in personality between marketing coordinators at mid-western hotel chains and marketing coordinators in general. With all stated caveats, we can help clients benchmark their employees on characteristics that matter for performance, as long as it is a worthwhile endeavor and we have the data to do it. Even then, we should not be creating stand-alone graphs or tables that simply plot group averages against each other. We must tell the story around and behind the graph or table to focus attention appropriately. Only then can a personality benchmark be meaningful, impactful, and actionable.

Personality has been one of the hottest trends in assessment over the last 10-15 years, as organizations and practitioners realize the value and utility a personality can provide in selecting and developing talent. Witness the rise of numerous personality instruments in the marketplace, the use of personality to develop the most highly-prized organizational talent, and the role of personality in the red-hot topic of derailment. While everyone at Hogan certainly would agree whole-heartedly with this movement, we’ve also been banging our drum about the role of values and culture in organizational performance. Despite ample evidence (both scientific and anecdotal), values just don’t seem to get the same degree of attention from organizations and practitioners.

This is a shame, because there is a lot of utility in these types of instruments, and the return on investment for assessing culture is tremendous. In a recent example, one of our financial services clients used the Motives, Values, Preferences Inventory (MVPI) in an effort to reduce turnover in one of their frontline positions. After one year of using the MVPI, they had reduced their turnover by 66%. That’s not a typo – two-thirds reduction in turnover!

I’m not arguing that we should abandon personality and narrow our focus on values. Quite the opposite. Personality and values are very distinct constructs, and each adds incremental prediction and validity over the other. Personality has a lot to do with our abilities to perform certain types of tasks, while values have more to do with our motivation and satisfaction with an organization’s culture. An employee whose values align with the organization’s culture will be more satisfied, and likely to work harder and with a better attitude. If the values and culture don’t align, then even a very capable performer won’t be motivated to do his or her best.

Going back to the study above, we were able to find significant reductions in both voluntary turnover (employees who wanted to leave the organization) and involuntary turnover (employees who were shown the door). Voluntary turnover is naturally where we would expect to find the biggest impact; satisfied employees won’t be searching Monster.com on their lunch break. But the reduction in involuntary turnover means that the organization had employees who were capable of doing the job but just didn’t want to, and were subsequently being terminated for poor performance. By aligning the culture and values for these employees, these underperformers improved as a result of increased motivation.

So how much is this all worth to an organization? In the study above we estimated the cost of turnover at half an employee’s annual salary (a conservative estimate). The annual savings attributed to this program (the difference in the number of employees turning over each year) is well into the millions. Annually. With an ROI of greater than $30 for every $1 spent, this program has paid for itself over and over and over again.

Jarrett Shalhoop
Senior Consultant
Hogan Assessment Systems

Because items on personality assessments have no correct or incorrect answers, users must apply norms to interpret scores. Norms exist largely behind the scenes, and aren’t as sexy as other issues. As a result, they receive little attention relative to other issues, and many do not consider their importance. However, norms can determine whether a person gets a job, receives admittance to academic programs, or is deemed clinically “disordered.” These examples show why test takers should care about norms, but why should test users care?

The answer is simple – validity. Put simply, validity concerns the accuracy of assessment-based decisions, or whether the instrument does what it’s designed to do. Because norms inform decisions based on a person’s score, they impact validity. As an example, consider two assessments designed for adult working populations. The only difference between the two instruments is their norms. Norms for one instrument come from an anonymous sample of Internet users. Norms for the other assessment come from working adults sampled based on labor force and demographic estimates. Because both assessments are intended for adult working populations, the validity of decisions based on these assessments depends on how well the scores reflect the intended audience. Here, norms for one assessment give users confidence in making decisions based on an apples-to-apples comparisons; norms for the other do not.

Hogan takes great pride in the quality and continued accuracy of our assessment norms. We continuously monitor our norms, completing incremental updates as necessary. More importantly, we obtain proportionate representation across occupational and demographic categories to make certain that our norms accurately reflect intended audiences. Internationally, we also develop local norms to give our clients and partners confidence that results reflect intended audiences in local populations. Collectively, these efforts ensure that Hogan remains an industry-leader in assessment norming practices.

Blaine Gaddis
International Research Manager
Hogan Assessment Systems

Because items on personality assessments have no correct or incorrect answers, users must apply norms to interpret scores. Norms exist largely behind the scenes, and aren’t as sexy as other issues. As a result, they receive little attention relative to other issues, and many do not consider their importance. However, norms can determine whether a person gets a job, receives admittance to academic programs, or is deemed clinically “disordered.” These examples show why test takers should care about norms, but why should test users care?

The answer is simple – validity. Put simply, validity concerns the accuracy of assessment-based decisions, or whether the instrument does what it’s designed to do. Because norms inform decisions based on a person’s score, they impact validity. As an example, consider two assessments designed for adult working populations. The only difference between the two instruments is their norms. Norms for one instrument come from an anonymous sample of Internet users. Norms for the other assessment come from working adults sampled based on labor force and demographic estimates. Because both assessments are intended for adult working populations, the validity of decisions based on these assessments depends on how well the scores reflect the intended audience. Here, norms for one assessment give users confidence in making decisions based on an apples-to-apples comparisons; norms for the other do not.

Hogan takes great pride in the quality and continued accuracy of our assessment norms. We continuously monitor our norms, completing incremental updates as necessary. More importantly, we obtain proportionate representation across occupational and demographic categories to make certain that our norms accurately reflect intended audiences. Internationally, we also develop local norms to give our clients and partners confidence that results reflect intended audiences in local populations. Collectively, these efforts ensure that Hogan remains an industry-leader in assessment norming practices.

Blaine Gaddis
International Research Manager
Hogan Assessment Systems

It’s surprisingly easy for busy organizations to overlook the importance of fundamentals, even when it comes to something as critical to reputation and profitability as safety. And safety starts with talent management.

While safety training certainly helps, what organizations need most are employees who think and act safely in the first place. If an organization doesn’t have proper insight into how individuals contribute to the safety climate, even the most extensive program will deliver limited success.

Continue reading the article "Building a Safer Climate Through Talent Management."

It’s surprisingly easy for busy organizations to overlook the importance of fundamentals, even when it comes to something as critical to reputation and profitability as safety. And safety starts with talent management.

While safety training certainly helps, what organizations need most are employees who think and act safely in the first place. If an organization doesn’t have proper insight into how individuals contribute to the safety climate, even the most extensive program will deliver limited success.

Continue reading the article “Building a Safer Climate Through Talent Management.”

It's been awhile since we've posted something new. We've been taking some time to get our bearings and to discuss some new possibilities for this blog space, beginning with some different voices and and expanded content scope.

To kick things off, we're linking to a recent article on changeboard.com that discusses the use of Hogan assessments in preparing participants for 2009's Rivers of Ice expedition in Patagonia.

While it's true that Hogan's assessment tools were primarily developed for (and are associated with) workplace applications, they've proved extremely useful over the years in a variety of "extracurricular" environments ranging from professional athletics to reality television to high adventure.

From the changeboard article Applying the Hogan tests to extreme conditions:

The 2009 Rivers of Ice expedition was the first attempt at an unsupported crossing of the Southern Patagonian Ice Cap by a team of two adventurers, Tarka L´Herpiniere and Katie-Jane Cooper. For many different reasons and on many different levels this was to be the most extreme and challenging expedition for both of them.

As their coach Sarah Fenwick (a Getfeedback Associate) worked with them on tools and techniques to maximise their strengths and minimise the potentially dysfunctional aspects of their personalities that when stressed, tired, cold, hungry, etc. might potentially have eroded the quality of their relationship, or worse, derailed the expedition.

Read more.

It’s been awhile since we’ve posted something new. We’ve been taking some time to get our bearings and to discuss some new possibilities for this blog space, beginning with some different voices and and expanded content scope.

To kick things off, we’re linking to a recent article on changeboard.com that discusses the use of Hogan assessments in preparing participants for 2009’s Rivers of Ice expedition in Patagonia.

While it’s true that Hogan’s assessment tools were primarily developed for (and are associated with) workplace applications, they’ve proved extremely useful over the years in a variety of “extracurricular” environments ranging from professional athletics to reality television to high adventure.

From the changeboard article Applying the Hogan tests to extreme conditions:

The 2009 Rivers of Ice expedition was the first attempt at an unsupported crossing of the Southern Patagonian Ice Cap by a team of two adventurers, Tarka L´Herpiniere and Katie-Jane Cooper. For many different reasons and on many different levels this was to be the most extreme and challenging expedition for both of them.

As their coach Sarah Fenwick (a Getfeedback Associate) worked with them on tools and techniques to maximise their strengths and minimise the potentially dysfunctional aspects of their personalities that when stressed, tired, cold, hungry, etc. might potentially have eroded the quality of their relationship, or worse, derailed the expedition.

Read more.