Neuropsychological impairments that everybody has

In recent months, several US employees working at the US Embassy in Havana have reported subjective complaints including headaches, weakness and fatigue, vertigo, hearing loss, insomnia, memory problems and loss of concentration. They invariably linked these symptoms to acoustic stimuli of varying intensity and duration, experienced whilst in Havana. The story, which has precipitated an ongoing diplomatic standoff between the US and Cuba, has been branded by the media as evidence of a malign 'sonic attack'.

The reports of these symptoms are largely anecdotal, but apparently stronger objective data have now emerged, in a high-profile paper with a supporting editorial, in the Journal of the American Medical Association. The paper reports on 21 people, all US diplomats previously working in Havana, concluding that they exhibit neurological manifestations of unknown origin, which resemble mild cerebral concussion, yet with no evidence of brain injury, and possibly related to obscure acoustic stimuli. The evidence for this extraordinary claim rests in large part on the extensive neuropsychological testing of six of these 21 people, with significant cognitive impairments reported in all cases.

This article has attracted a lot of attention, but remarkably little scrutiny. A closer reading of the paper shows that the neuropsychological evidence presented in the Supplementary Material (eTable 2) is almost unbelievably flimsy. The six people assessed with the neuropsychological battery were tested with 37 tests split across 10 cognitive domains. In each domain, the person was considered impaired if their score fell below the 40^th (fortieth) percentile in at least one of the tests. This threshold for impairment is inexplicably high.

By definition, 40% of people assessed on any test should score at or below the 40^th percentile. Considering that each participant was given 37 tests, without any statistical correction for the number of tests administered, it seems unlikely that anyone would escape with a clean bill of cognitive health (for further analysis, see Della Sala & Cubelli, in press). As we have reported in a letter to the Journal of Neurology, we tested this intuition using a simple simulation. We substituted the authors' raw data for randomly-generated test scores, and applied their bizarre diagnostic criteria (Further details of this simulation, including analysis code and graphical output, are available at https://osf.io/5b4sq/). Across a thousand repetitions of our simulation, the cognitive symptoms of this syndrome proved to have a worrying lack of specificity: everybody is affected.

It is hard to understand how claims like this, based on the most blatant p-hacking, could pass any meaningful peer-review process. The pseudo-scientific approach taken implies that one may define arbitrarily liberal cut-offs to make one's case, and interpret the outcomes as if they were clinically valid. We should be much more worried about reputational damage to neuropsychology, and psychology in general, than about any sinister new sonic weapons.

Robert D McIntosh, PhD and Sergio Della Sala, MD, PhD;

Human Cognitive Neuroscience, Psychology, University of Edinburgh, Edinburgh, UK

Reference

Della Sala, S., & Cubelli, R. (2018). Alleged "sonic attack" supported by poor Neuropsychology. Cortex, in press.

Della Sala, S. & McIntosh, R. D. (2018). Cognitive impairments that everybody has. Journal of Neurology.