One in ten student research participants don’t make an effort

It's near the end of your university semester, you're tired and now you've got to sit through 90 minutes of monotonous psychology tests to fulfil the requirements for your course. This is a familiar situation for psychology undergrads, many of whom form the sample pools for thousands of psychology studies.

Concerns have been raised before that psychology findings are being skewed by the (lack of) effort students put into their performance as research participants. Last year, for example, researchers found that students who volunteer near the end of term perform worse on psychology tests than those who volunteer earlier.

Now Jonathan DeRight and Randall Jorgensen at Syracuse University have investigated student effort in 90 minutes of computerised neuropsychology tests designed to measure attention, memory, verbal ability and more. The session, which took place either during a morning or afternoon late in the Spring semester, involved the students taking the same broad battery of tests twice, with a short gap in between. The students received course credits for their time.

To test whether the students were making a proper effort, the researchers embedded several measures – for example, performing worse than chance on a multiple-choice style verbal memory challenge was taken as a sign of low effort; so was performing more slowly on an easier version of a mental control task than on the more difficult version.

Among the 77 healthy student participants who took part (average age 19; 36 women), the researchers identified 12 per cent who failed at least one of the embedded measures of effort during the first battery of neuropsych tests; 11 per cent also failed one or more measures during the second battery. The vast majority of those who showed low effort had participated in the morning. In fact, focusing only on the morning participants, one in four displayed low effort.

Unsurprisingly, low effort also went hand in hand with poorer performance on the neuropsych tests, especially one of the longest and most dull cognitive tests (the "continuous performance task"), and especially during the second battery. A consistent exception was a particularly complex version of a test of mental self-control (the Stroop task) – perhaps because the challenge of the task provoked more concentration, even from students who were mostly not trying hard.

The estimate from this study of the fraction of student research participants not making an effort are consistent with some prior studies, but not others (the latter research found less evidence of poor effort). Clearly more research is needed. DeRight and Jorgensen concluded that "healthy non-clinical samples cannot necessarily be assumed to have put forth adequate effort or valid responding." They added: "Assessing for effort in this population is imperative, especially when the study is designed to provide meaningful results to be used in clinical practice." This last, important point is a reference to the fact that results from students are often used to establish estimates of "normal" performance on neuropsychology tests, for comparison when investigating patients with brain damage or other problems.

Further reading

DeRight, J., & Jorgensen, R. (2014). I Just Want My Research Credit: Frequency of Suboptimal Effort in a Non-Clinical Healthy Undergraduate Sample The Clinical Neuropsychologist, 1-17 DOI: 10.1080/13854046.2014.989267