In this installment of our FAQ series, the Hogan Research Department (HRD) tackles common questions on Hogan’s rigorous norming and reliablity procedures. Add your questions in the comments below and stay tuned for our next blog on the Hogan Personality Inventory.
NORMS
Q: What are the normative group sizes for each of the inventories?
A: Unlike other assessments that are normed on samples of only a few hundred cases, Hogan collects comprehensive normative data to ensure that our assessments accurately represent intended populations. However, for summary purposes, the global normative sample sizes for each of the Hogan assessments are as follows:
HPI: N = 144,877
HDS: N = 67,614
MVPI: N = 48,267
Q: What is the composition/size/basis of the norm groups?
A: For all assessments, Hogan conducts stratified random sampling to ensure that assessment norms represent intended populations. To create accurate norms, Hogan obtains workforce estimates from relevant sources (e.g., Department of Labor, Equal Employment Opportunity Commission), and samples normative data to match these estimates. In this way, Hogan ensures the norms apply across occupational (e.g., job families, industry sectors), demographic (e.g., age, gender, and ethnic groups), and other (e.g., selection vs. development application) groups.
Q: Do you/can you break out more specific/local norm groups? Can you break out norms by industry?
A: Although Hogan’s general norms, representing the working population, are relevant for most selection and development applications, some clients request more specific norms for population subgroups. For example, a client may want to compare an individual’s scores to other managers in a specific industry or, perhaps, even a specific organization or location. For such applications, Hogan can create subgroup norms applying the same considerations of composition and relevance used to develop the general norms. Hogan requires the subgroup’s normative dataset be sufficiently large and diverse in its representation. For example, to construct a managerial norm, Hogan requires a minimum of 500 cases of managerial data representing multiple organizations and industries in
the target location. However, sample composition requirements vary as a function of the referent population. Other examples include regional, industry, and client-specific norms and may be developed with sample sizes as small as 100 individuals in some cases. When developed appropriately, these norms can provide useful supplemental information to more general norms and are especially helpful in developmental contexts. Although these specific subgroup norms prove useful as supplemental desktop reference materials, in producing reports for score interpretation, Hogan does not use these subgroup norms as replacements for more comprehensive general
workforce norms.
RELIABILITY
Q: Do scale scores change over time? If so, which ones? How often should one re-take inventories to capture “current functioning”?
A: On an individual level, research shows that personality is stable. Although an individual’s scores may fluctuate slightly over time, significant shifts in HPI and HDS scores are rare and usually the result of careless responding. However, on an aggregate level, the average levels of HPI and HDS scale scores may change over time. For example, the average level of the HPI Sociability scale may be higher now than 20 years ago, indicating that, on average, people are more social now than then. To account for these changes, Hogan consistently monitors and maintains assessment norms to ensure that they reflect current functioning in intended populations. Because personality is stable, and because Hogan maintains norms to reflect current functioning, client respondents need not retake inventories.
Q: Are there age-related or generational changes in scale scores?
A: Collaborative research between Hogan and Peter Berry Consultancy examined differences in scale scores between different countries, as well as between different generations. To briefly summarize, researchers examined scale score differences between Baby Boomers (born 1946 – 1964), Generation X (born 1965 – 1977), and Generation Y (born 1978 – 1994). Although research revealed small generational differences in scale scores, these differences are not significant in practice. At an individual level, these results indicate that one cannot assume anything about a person’s personality or value set simply by knowing when they were born.