The concept of validity is at the core of any personality assessment. There are many approaches to developing a valid assessment, and unfortunately, in an industry with very little regulation, validity varies greatly between providers. From the user’s perspective, validity should be measured simply by the assessment’s ability to accurately predict what the publisher claims it will predict.
Of course all valid assessment providers publish in-depth technical manuals describing the assessment’s conceptual background, underlying construction, validity, normative sample, etc. So, my first recommendation would be to consult the assessment provider’s technical manual. While the technical manual should provide scientific foundation and alleviate any validity doubts, I often challenge clients and prospects to consider other components of the validity equation when exploring assessment providers. Take a closer look at the assessment provider itself and consider a few relevant points:
1. Who’s the publisher? More specifically, who created the initial items in the assessment and how are they maintained? Do they have any IO Psychologists on staff? Can you pick up the phone and speak directly with the author of the assessment? Is the assessment the core of the provider’s business, or is it just one of a thousand different products offered? What contributions have the publisher made to the scientific advancement of personality assessments?
2. Longevity? How long has the assessment been in use? Often times the size of the normative sample will provide insights to the longevity of the assessment. Has the assessment ever been legally challenged? Does the assessment provider have a solid track record and reputation in the industry? Do they have an archival database of assessment results and criterion data?
3. Purpose? Trace back the foundation of the instrument to its original intent; was the assessment designed to be used in the business world to predict business outcomes? Strategic self-awareness is only relevant if it is tied back to real business challenges and situations.
4. Relevance? Will the assessment provider’s offerings deliver the content necessary to meet the goals and objectives of your assessment initiatives? If an assessment provides nothing but glowing compliments to participants, it creates a false sense of validity; who is going to argue with an assessment that tells us how great we are? Do the constructs measured provide both strengths and weaknesses?
5. Depth of content? Does the provider publish multiple assessments to capture different aspects of personality? Can those assessments be combined to provide a holistic approach to managing talent? Does the provider restrict administration of their assessments to those qualified to interpret and understand the results? Does the provider offer a certification process?
6. Global Mindset? Has the assessment been adapted to today’s global business climate? Does the provider have a true global normative sample? Can you be assured participants are being measured with the same yardstick across borders? Do they offer local support for your global initiatives?
With over 3000 domestic assessment companies in the marketplace, due diligence is critical when selecting a provider. Any assessment provider can create clever marketing propaganda simplifying the assessment validity equation to a few bar charts, coefficients, or feel good comments. Evaluating your prospects according to the guidelines above, however, will provide a much clearer picture of whether or not those assessments will perform as advertised.
