What's inside a psychometric test?

You may have been asked to complete a test or questionnaire and wondered where these exercises come from, why they are used and whether they are used properly.

Test and questionnaires are used to find out about a person’s capacities, work style or values. Employers need this sort of information when they want to recruit a new employee or understand the potential and development needs of an existing one. Individuals need to consider their own abilities and personal style when making career choices, so careers advisers are also frequent users of tests.

The research evidence consistently shows that objective measures such as tests and questionnaires provide more accurate assessments of individuals than subjective approaches such as interviews or evaluating CVs. While these subjective approaches can provide useful information, particularly in skilled hands, the reliability and precision of objective measures is difficult to match.

In the current climate, where an employer can receive thousands of applications for a single position, computer administration of questionnaires and tests provides a way to evaluate large numbers of people and identify the best prospects to invite to interview and further assessment. Without such tools a recruiter would be faced with spending days going through a pile of CVs.

Tests and questionnaires are often referred to as ‘psychometric’. That is because psychological theories of human behaviour and its measurement have been used in their construction.

When developing a new psychometric measure, psychologists first carefully define what it is they want to measure. Often this involves researching the evidence on work performance to identify which personal factors are related to quality of functioning in a particular area.

For example, a concern for detail is important for a proof-reader but may be less desirable in a tour guide – who might bore listeners with minutiae. The tour guide would benefit more from an extroverted personality style when interacting with tourists. When people’s natural capacities suit their job, they tend both to enjoy their work more and perform better.

Once the characteristics that need to be measured have been determined, the next task is to consider how best to measure them. This may seem quite straightforward. If you want to know about a person’s ability to reason with numerical information, then the simplest approach is to construct some problems which require the analysis of numerical information to solve.

However, in writing a test it is important to be careful not to require irrelevant skills which could introduce bias into the test results. For example, if numerical problems are described in long paragraphs of complicated text, someone who is good at reasoning with numbers but has a poor vocabulary may not understand the questions and gain a poor score as a result. The skill of the test developer is to ensure that the test only requires relevant skills.

When a pool of questions has been created, they are thoroughly reviewed before piloting with large groups of people. The data from the pilots are used to assess and refine the accuracy of the test (its reliability in psychometric jargon) and to ensure that it is working fairly and measuring what was intended (the test’s psychometric validity). The pilot can also provide standardisation information, which helps in interpreting the results from the test – for example, how typical applicants score on the test and what score level is appropriate for different roles.

After all this work the test is ready for use. It may have been developed for a specific organisation to use – for example the army has a set of tests they use to evaluate potential soldiers – or it may have been developed by a test publisher for sale to a range of users. 

It requires appropriate skills and training to use tests. The British Psychological Society has defined standards of competence that a person should meet before being able to purchase and use tests and questionnaires. You can check whether a person has appropriate qualifications to use tests through the PTC Register.

Good test users use only instruments that have proven efficacy for the task and follow the ethical guidelines for their use. They need to be able to identify what types of tests and questionnaires would provide the information they are looking for and to evaluate the quality and research evidence supporting different measures to be able to choose one that will be most effective. They need to consider how the scores will be interpreted and what use will be made of them. Test users also need to provide people asked to take the test with appropriate information, so they can make an informed decision as to whether to do so. 

For example, a test taker needs to know in general terms what the test will show about them and how the information will be used. The test user needs to make every effort to ensure that the test is taken under appropriate conditions and that the taker receives feedback on the results if he or she wants. The British Psychological Society publishes a number of guides outlining best practice in the use of tests.

Test results are considered personal information under the Data Protection Act and therefore a test taker has right to receive information on their results. Test scores are just a string of numbers and so won’t mean anything to someone not trained in the use of the test. Feedback to the test taker should therefore explain the meaning of the results in context. If you have been asked to take a test or complete a questionnaire you can request some feedback on your results if this was not already supplied.

It is this work behind the scenes that ensures that responses to what may seem like an arbitrary set of questions can yield useful information about what a person can do and how they might go about a task. It allows the users of psychometric tests to make more effective decisions about placing people in suitable roles or providing them with appropriate development opportunities.