11 Ways to Judge the Accuracy of Assessment Tools

Did you know that in the 1950s—in an effort to determine whether soldiers were mentally fit to handle deadly weapons—psychometric testing was created by the U.S. Military? (Actually, a more accurate term would be ‘pathology’ tests.) In the years that followed, a few enterprising entrepreneurs decided these pathology tests (with minor adjustments) could provide them with valuable insights into how people work together to bring about common business goals. After fifty years of testing, we now know the level of accuracy of these tests. We know where they are helpful and where they fall short. How can YOU know if the assessment you are using measures up? That’s the goal of this report—to provide you with 11 Key Questions you can ask yourself to determine what tool you should use in your recruitment process and to determine the true accuracy of any hiring assessment.

1: What ‘norm’ are your job candidates/employees being compared to?

Most assessments measure personality and behaviour.

Here’s the simple version of how personality and behaviour (including situational judgement) based tests are created. As you will see, created is the key word here. These profiles are created by psychologists who observe the behaviour of a particular group of people. They then create questions designed to show where a person scores in relation to the observed ‘norm’ of that group. In other words, their goal is to match the questions they’ve created, to fit the ‘norm’ they are creating. Every assessment tool in the market place is “culture” driven. When an assessment tool is to be taken in a different culture, they must, again find the standard of measure, or ‘norm’, for that culture. While this process has some value, there is a massive element of subjective bias involved. Clearly, these tools are affected by many factors including the geography, culture, income, age or gender that group of individuals. Just know this ... subjective bias significantly lowers the accuracy of these tests. Perhaps this is why, after 70 years of trial and error in the hiring process, research clearly shows that typical assessment tools are just not accurate. Business leaders find it difficult if not impossible to trust the typical hiring assessment. Have you found this to be true for you?

Is there an alternative...?

In the years before the outbreak of the Second World War an amazingly accurate way to measure performance in people was discovered by the scientist and philosopher Robert S. Hartman. He was nominated for a Nobel Prize for his research in 1973. His approach was NOT based on simply observing behaviour or personality. Hartman’s theory was that we all think and make decisions in the same way—that there are three core decision making dimensions. He called this, “The Structure of Thought.” Hartman spent the next 40 years of his life proving his theory scientifically (not the few days or hours it takes to establish a new norm with other tools…). He proved his theory with a totally objective mathematical formula. This formula measures how people think and how they make decisions. This was a major discovery because unlike “subjective norms” that come as a result of observing behaviour and personality, Hartman discovered that the structure of thought has its own single norm. This norm is the same for people all over the world, regardless of culture, geography, income, age, gender, or other factors. We all have the same structure of thought. Hartman didn’t create subjective norms. His science validation confirms a norm – a single objective norm that already exists in all of us. That's extremely important because when the measurement is totally objective - you can trust its accuracy! The Axiometrics™ profile scientifically measures how you think and how you make a decision - what Hartman called your ‘structure of thought’.

It directly measures skills, competencies, attitudes, talent, whether or not you are accessing your talent, and much more. The Axiometrics™ profile will quickly identify precisely who will and will not perform in a company with greater accuracy than all other human metrics systems. Clients report 90-100% accuracy in filtering out those who will not perform in an organisation’s unique culture.

2: Does the questionnaire require you to answer questions about yourself?

Are you totally objective when answering questions about yourself, or like most of us, a bit biased and subjective? Why is this important? Just consider how bias enters into the equation when your job applicant takes the assessment. If they are asked to describe themselves, you can be confident the accuracy will be negatively affected. With the Axiometrics™ profile, you are never asked to describe yourself. You are never asked to describe what you would, or might do given a set of circumstances. And you are never asked what others might expect you to do - which is a way of getting you to describe what you’re “really like.” Actually, there are no questions at all – there are 18 phrases, or words. You simply rank them from best to worst. The Axiometrics™ Profile is not subjective. You are not asked to describe yourself and you are not asked to describe how you would perform. When you take the Axiometrics™ Profile, you are actually performing a task - making it a totally objective measurement.

3: Does the questionnaire telegraph what it intends to measure about you?

Obviously you get greater accuracy when the person taking the test doesn’t know what the profile intends to measure. Because given the chance, applicants are likely to try to fool the test to make themselves look better. The Axiometrics™ profile is unique. There is no obvious relationship to any of the 18 items to be ranked. They appear totally random and unrelated. Because it is impossible to guess how you are being measured, people are amazed at the accuracy of the feedback.

4: Does the tool measure before or after a decision has already been made?

Measuring personality, behaviour, or style, is like taking a digital photograph of a person. You can measure them on the outside—but you can only guess what's really going on inside. It’s like judging a book by its cover. Observing behaviour may give you some clues about how an applicant might fit certain roles, however it’s what personality and behaviour doesn’t tell you that causes expensive hiring mistakes. Mistakes you would never make if you could only peel back the cover and the layers and see what’s going on inside. What if, instead of a digital photo, you had an X-Ray? What if you could really see what was going on inside? What if you could directly measure exactly what you need to know about someone? The Axiometrics™ profile is this X-Ray, enabling us to identify the internal valuing systems that influence our perceptions, decisions and actions - allowing us to clearly understand "why" we do what we do! Performance is all about making the right decisions! To make a decision, your mind takes four critical steps which are: Perceive, Associate, Analyse, Decide.

Process_From_How_we_Value_to_ActionMK.jpg
Behavioural assessments measure only after a decision has already been made. This misses the hidden variables - the real reasons a person will or will not, or can or cannot perform. The Axiometrics™ profile measures what’s going on before the decision is made. It actually measures how your job candidate thinks and how they make a decision. Failing to measure all four of these Critical Variables means you’ll be missing crucial information that would enable you to accurately predict job performance.
 

5: Does your assessment measure only personality and behaviour?

It’s important to know if the tool you are using will give you the feedback you need to make the right hiring decisions. Some tests will help you understand how people might work together. What we’re focused on here though, is hiring and selection. As you well know, behaviour and personality are susceptible to change.

Often behaviour is affected by mood, emotion or circumstances. On the other hand, in the short term, how you think and make decisions (your thought structure) does not change. This is the key to understanding why you cannot fully trust any measurement which is susceptible to current moods, emotions or circumstances.

6: Does it answer the specific questions you need to know?

To make great hiring decisions, what else do you need to know? Here’s a small list... Have they got the required skills and competencies? Can they do the job? Will they do the job? Can they do the job in a specific business environment? How clear is their thinking and quality of judgment? Have they got the right attitude? Lastly, and of greatest importance, do they have access to their talent?

Many existing profiles fail to measure the crucial questions above, especially in relation to a candidate’s inner motivations.

The Axiometrics™ profile directly measures all of these and more. Our unique technology measures and alerts you to the direct risk you face with every job candidate. Although only recently introduced to the UK, our highly accurate technology has for over 15 years been identifying 8 out of 10 risky hires for some of the US's top companies, saving them millions of dollars - in some cases, in a few short months.

7: Does it work in every culture?

The majority of psychometric tools have to have a new norm created every time they are used in a new environment and culture. The reference groups for this are often very small, leading to low rates of accuracy. Remember...everyone in the world is susceptible to current moods, emotions or circumstances and that will affect how they approach an assessment.

8: Does it discriminate in any way?

You need to know that if the technology you’re using is discriminatory in any way, it is illegal. The Axiometrics™ profile asks you to perform a task that has no connection to race, gender, age, creed or any other socio-cultural classification. There is no difference in scores.

9: Does it invade a person’s privacy?

Many feel that being asked personal questions is an invasion of privacy. You do need to be careful as the majority of psychometrics test use personal questioning.

The Axiometrics™ profile does not ask personal questions. In fact, as discussed earlier, there are no questions at all. There is just the ranking of what seem like random, unrelated statements.

10: Does the feedback identify when a candidate has tried to “fool” the system?

Most systems have a limited process for identifying false answers. Tests try to guard against intentional manipulation by posing the same question more than once but wording it differently.

The Axiometrics™ profile software is specifically programmed to indicate whether someone tries to manipulate the instrument. When the system detects a flawed or manipulated test, the system is designed to give you a “no report.”

11: Have validity studies been completed and are they available to you?

There are FIVE types of studies designed to confirm validity:

  • Construct Validity: Has the instrument been constructed properly?
  • Adverse Impact Studies: Could the instrument adversely impact participants?
  • Co-relational Studies: How does it relate to other assessments?
  • Criterion Studies: Does it measure the criteria it is designed to measure?
  • Face Validity: Do those who have experienced the technology believe in its accuracy?

All five validity tests have been completed on the Axiometrics™ profile. Most importantly, the system behind the Axiometrics™ profile has been validated over and over again in unique business environments. Some of these studies were conducted with Dollar General's entire workforce – over 40,000 people, from the President Cal Turner in Nashville, to the warehouse employees in Miami. Studies have also been conducted at Sara Lee Corporation, GTE, CUNY Mutual, Arthur Andersen, HCA, AT&T, Harvard, Princeton, Yale, University of Chicago and many others worldwide.