The ways that student samples differ from the public varies around the world

Biologists have their fruit flies and rats, psychologists have students. An overwhelming amount of behavioural science is conducted with young people at universities on the assumption that it's safe to generalise from this species of human to people more generally. There are some common-sense reasons for thinking this might be a problem and also some more specific issues, which we've documented before, such as that burnt-out students could be skewing the findings.

Now a recent study in PLOS One shows that the ways students differ from the public is different depending on which country you're in, meaning it's extra complicated to figure out if and when it's appropriate to extrapolate student-based findings to people as a whole.

The researchers Paul Hanel and Katia Vione at Cardiff University used data collected from over 86,000 people (including 6,352 students) in 59 countries as part of the World Values Survey.  They wanted to see if students differed from the general population on various measures, such as trust in strangers, confidence in democracy, morality and personality, and if so, whether this was fairly constant across countries or varied according to whether a country was more individualistic or collectivist (either of which would make it easier to anticipate and account any for student-public differences when conducting research).

The researchers indeed found some notable differences between students and the general public, but perplexingly these differences often varied between countries and they mostly weren't explained by the key country characteristic of being individualistic vs. collectivist.

For example, in New Zealand, students showed heightened respect for the elderly compared with the public, but in Australia, the opposite was found. In China, students showed more confidence in political institutions than in the public, while in Germany, students showed less confidence. Students in the USA saw dishonest behaviours, such as stealing, as more justifiable than the public, but in India, students saw such acts as less justifiable than the public.

It was a similar story for personality: for instance, in Colombia students tended to score higher on Conscientiousness than the public, but in Brazil, the reverse was true. However in a comment posted online after the publication of the research, one of the authors said it had come to their attention that the personality measure used in the Moral Values Survey (the BFI-10) was unreliable, and so the results for personality should be disregarded, but he added that "the other seven variables included in our analysis are reliable and can be interpreted as described in the article".

In their paper, Hanel and Vione said their results "further support the claim that generalizing from students to the general public within personal and social psychology is problematic, at least while we do not know what predicts those differences."

These new findings are somewhat disconcerting, but studies of this kind are also very welcome given that psychology's dependence on student samples is unlikely to end any time soon. Crucially, the more we know about how students tend to differ from the public, the better placed we will be to interpret new student-based findings in an informed and considered way.

Correction: On 7 April 2017, this article was amended in light of one of the study author's online comments at PLOS that the personality measure used in the research was unreliable (thank you to Frank O'Connor for bringing this to our attention).