On measurement, meaning and myth

The first piece of advice that my PhD supervisor (Wilmar Schaufeli) offered me was that the easiest concepts to measure were not the most interesting, and conversely the most interesting concepts were not the easiest to measure. I found myself returning to this thought recently, as the gap between what we measure and its meaning is diverging at an alarming rate.

I noticed that my students and non-academic collaborators accept certain organisational ideas and concepts as hard facts, which can lead to some very sloppy thinking (at best) and dysfunctional conclusions (at worst). We, the academics, are largely to blame – we have been bystanders to these runaway trains. Myths and half-truths speed along on one track, fuelled by social media, increasingly difficult to derail as they pick up speed.

Here, I examine two organisational concepts and a well-known model that seem to be easily hijacked as myth trains, taken as 'gospel' by my students and the wider community: job burnout, psychological safety, and the Job Demands-Resources Model. The scientific limitations of all three have been well covered in the literature: I am more interested in how all three have been diluted down and fed into industrialised mass-market ideologies, rather than being treated as ideas that need to be chewed upon constantly.

Research for research's sake

We would do well to remember that 'All scientific knowing is indirect, presumptive, obliquely and incompletely corroborated at best' (Campbell, 1975, p.1120). And yet every time I invite my students and collaborators to examine the original papers on job burnout, psychological safety, and the Job Demands-Resources Model, I see the anxiety produced by the epistemic realisation that such concepts are fragile and are intended as starting points not end ones.

This anxiety perhaps stems from and reflects the metaphysical obsession that psychology has with method (Hughes 2023; Robinson, 1995). Measurability usually trumps meaningfulness. Indeed, this lack of meaning is starkly highlighted by research showing that many citations serve rhetorical functions and reflect little-to-no influence on the citing authors (Teplitskiy et al., 2022).

Congruently, it appears there is a growing recognition that theoretical contributions represent 'window-dressing' in management and organisation journals (Prasad, 2023). The past-President of the Academy of Management, Professor Herman Aguinis, published a paper with several colleagues in 2021 lamenting the fact that the overwhelming majority of articles in organisational behaviour and human resource management (OBHRM) do not have policy implications and thus, according to the authors, risk becoming 'societally irrelevant'. The academic papers remain rarely read outside of academic, certainly in comparison with the social media channels which may spit out diluted versions for the wider community to consume with gusto.

Moreover, such social irrelevance is fed by the ongoing 'publish-or-perish' environment that is affecting both paper quality and early career wellbeing. For example, PhD students are forced into questionable authorship practices, and they appear to be reinforced through a combination of coercive power relations and dominant norms in some research cultures (Goddiksen et al., 2023). This begs the deeper question of whether we are doing research for research's sake (i.e., career progression and/or citation scores rather than a scholarly vision). It's reasonable to speculate that less meaningful work is symbiotically linked with increasing rates of anxiety, depression and burnout among academics (Paitaridou et al., 2023).

These trends will inform our discussion of how the concepts of burnout, psychological safety and the JD-R model are being launched into narratives about work without the necessary epistemic parachutes to soften the landing. Ignoring our epistemic roots in this way has consequences, both in terms of how we are viewed as scholars by society (externally) and our communal wellbeing as a scholarly community (internally).

Burnout

The study of burnout has become flooded with exaggerated claims, oversimplifications and wilful ignoring of the meaning of the concept. The idea that our knowledge should demonstrate 'warranted acceptability', as suggested by John Dewey (1938), is useful here. On the one side, Maslach and Leiter (2022) in their recent book on the topic, The Burnout Challenge, provide a sober, restrained, and realistic account of the phenomenon, that ticks all the boxes of warranted acceptability. However, this has not prevented the widespread heterogenous and ad-hoc use of their burnout measure (MBI) in the research literature (e.g. Doulougeri et al., 2016; Hewitt et al., 2020).

Burnout is a starting point, not an end one, an invitation to further research what factors are driving such feelings. However, it is used frequently as a 'diagnosis tool' by some academics, and an umbrella term for every ailment by the wider community. The cry to extinguish the burnout pandemic drowns out the more interesting question of how our work environments are continually designed to create conditions conducive to creating feelings of burnout (Montgomery, 2014; Montgomery et al., 2019).

Equally, the harsh reality is that we are still reinforcing the dysfunctional narrative of individual responsibility, with even the right to sit down at work being questioned (Demopoulos, 2023). In interventions studies, Burnout has been narrowed down to a stress 'thermometer' that needs to be reduced – much like a fever needs to be brought under control – with little consideration for the social, economic and organizational elements that created the toxic environment in the first place (Montgomery & Laindi, 2023).

It shouldn't be a surprise that the wider community treats burnout as a disease that needs to be treated and/or suggests it just another name for depression and exhaustion. Maslach and Leiter consistently advocate the need to carefully separate the 'wheat from the chaff' when researching burnout, but it seems some people ignore these warnings by taking a reductionistic approach to the problem (e.g., creating apps that will cure your burnout and improve wellbeing). The rush to diagnose and fix burnout contrasts with our limited understanding of the phenomenon, which includes its progression and resolution.

Psychological safety

Psychological Safety has matured as a core concept in organisational psychology, as the number and reach of studies has grown exponentially (Edmondson & Lei, 2014; Edmondson, & Bransby, 2023). To return to our warranted acceptability argument, Edmondson and colleagues tick all the boxes in terms of what can be known about psychological safety, and with laser-like precision they also pinpoint the limitations, which includes a significant lack of knowledge with regard to how to create psychological safety, the role that other team members play in creating in psychological safety, measurement in non-Western countries, and lack of longitudinal research.

However, Edmondson and colleagues' excellent assessment on the state-of-the-art on psychological safety is not reflected in the constant barrage of management articles encouraging us to use psychological safety as a tool to 'fix' productivity and increase organisational learning in the workplace. The best examples of such simplistic approaches can be found in Harvard Business Review articles (e.g., see Gallo, 2023; Ravishankar, 2022) which present 'clean' and unproblematic perspectives of the concept.

Much like burnout, shoving psychological safety down the throats of industry as an evidence-based solution to reducing employee silence and increasing voice is misguided. For example, there is evidence that psychological safety doesn't work in the expected way in healthcare (Lainidi et al., 2023; Montgomery et al., 2023), where figuring out what can be talked about trumps the generic idea of psychological safety. We run the risk of voice and psychological safety running in parallel, whereby speaking up about issues the organisation feels comfortable, which creates the illusion of psychological safety.

The job-demands-resources model

The job demands-resources (JD-R; Bakker & Demerouti, 2007; Demerouti et al., 2001) model is currently the most popular framework in occupational health psychology to investigate the relationships between job characteristics and employee well-being. Indeed, the most recent meta-analysis of longitudinal studies of the JDR indicates 'the JD-R model is an excellent theoretical basis to describe employee well-being in a broad range of organisations and occupational fields' (Lesener et al., 2019 p.96).

The use of word describe is very important in this sentence. I personally like the JD-R model and use it in my teaching, but I think the authors have flagged for us a reason to be more restrained in our excitement about the model. Moreover, Lesener et al. (2019) list the limitations of the model, which include the fact that the contributing studies are based on research that is self-reported; no study used objective measures of job demands or job resources; no comprehensive reasons why the studies chose specific time intervals; and the newer constructs (e.g. personal resources, job crafting) could not be examined due to limited availability of suitable studies.

In the realm of high difficulty/low difficulty studies, it would not be unfair to categorise the JD-R model as built on low difficulty studies (with the appreciation that all longitudinal research can involve considerable effort). Online methods and artificial intelligence are increasingly used to gather data quickly. More data, sophisticated statistics and multiple papers, an approach reinforced by journals conscious of impact factors and seeking papers that will be cited frequently. But again, it's at the expense of meaning. This 'MTurkification' of social and personality psychology has been called out by Anderson et al. (2019), who speculate that most of the giants of early social and personality psychology (e.g. Leon Festinger, Stanley Schachter, Kurt Lewin, Fritz Heider) would not be appreciated in today's academic job market, nor in the job market that has existed since the early 1980s.

We should be grateful for the hard work that has led to the JD-R model, but I think we could be humbler about its potential to deliver magic bullets in occupational health psychology. As Meehl (1990) said, 'Any working scientist is more impressed with 2 replications in each of 6 highly dissimilar experimental contexts than he is with 12 replications of the same experiment' (p. 111). Artificial Intelligence has made survey data collection easier and cheaper, but we risk the extinction of the kinds of difficult-to-conduct studies needed in some of the most important real-world problems that social psychology scholars have traditionally tackled (Anderson et al., 2019). Organisational psychology has been built upon high difficulty research (e.g. cognitive dissonance, the Hawthorne experiments, intergroup relations), and we need to remind ourselves that the usefulness of contemporary statistics (i.e. significance testing) 'would seem to be walking a tightrope between the serious pitfalls of over-determination and under-determination as two equally unfortunate end-products of the reduction of psychological phenomena to quantitative terms' (Brower, 1949, p.327). We are eager to share our knowledge with our students and the wider community of work, but we don't seem to be as eager to also share how everything is correlated with everything, more or less (the 'crud' factor noted by Meehl, 1990).

Back to our epistemic roots

I am mindful of appearing curmudgeonly about some great research. Organisational psychology is an applied pursuit, and it's vying for acceptability/usefulness in the wider arena of work. It's understandable that industry wants concrete tools to address problems and we (organisational psychologists) want to avoid appearing obscure. But we seem to have forgotten the 19th century warning of the pragmatic philosophers (i.e. Holmes, Dewey, James) that ideas are always provisional, and ready to be superseded by better ones. Put simply, if we don't acknowledge complexity and uncertainty – we shouldn't be surprised that the recipients of our knowledge become sceptical.

Forgetting our epistemic roots also runs the risk of replicating the dilution of management education as forensically charted by Khurana (2010) in his book From Higher aims to Hired Hands – begging the question of whether we risk becoming just the servants of industry. Social media has opened exponential routes for us to communicate more effectively with the wider community, but it has also led to the simplification of ideas that will ultimately degrade their worth and result in disengagement from individuals who will ultimately understand the limits of the concepts and theories.

Underestimating our audience is unwise, especially given the research showing how complexity is more desirable than we previously assumed (Grant, 2021). My recent collaboration experiences with professionals from health and social care indicates a fatigue with 'shiny' organisational interventions that promise more than they can deliver, along with a growing cynicism fuelled by psychology that seems to be deaf to the reality of food banks and economic insecurity.

• Anthony Montgomery is Professor of Occupational & Organizational Psychology at Northumbria University. [email protected]

References  

Aguinis, H., Jensen, S. H., & Kraus, S. (2022). Policy implications of organizational behavior and human resource management research. Academy of Management Perspectives, 36(3), 857-878.

Anderson, C. A., Allen, J. J., Plante, C., Quigley-McBride, A., Lovett, A., & Rokkum, J. N. (2019). The MTurkification of social and personality psychology. Personality and social psychology bulletin, 45(6), 842-850.

Bakker, A. B., & Demerouti, E. (2017). Job demands-resources theory: Taking stock and looking forward. Journal of Occupational Health Psychology, 22, 273–285.

Brower, D. (1949). The problem of quantification in psychological science. Psychological Review, 56(6), 325.

Campbell, D. T. (1975). On the conflicts between biological and social evolution and between psychology and moral tradition. American psychologist, 30(12), 1103.

Demerouti, E., Nachreiner, F., Bakker, A. B., & Schaufeli, W. B. (2001). The job demands-resources model of burnout. Journal of Applied Psychology, 86, 499–512.

Demopoulos, A. (2023, June 28).  'Not for employee use': why are US retail workers being denied chairs? . The Guardian.

Dewey, J. (1938). Logic. The Theory of Inquiry. New York, Henry Holt & Co.

Doulougeri, K., Georganta, K., & Montgomery, A. (2016). "Diagnosing" burnout among healthcare professionals: can we find consensus?. Cogent Medicine, 3(1), 1.

Edmondson, A. C., & Lei, Z. (2014). Psychological safety: The history, renaissance, and future of an interpersonal construct. Annu. Rev. Organ. Psychol. Organ. Behav., 1(1), 23-43.

Edmonson, A.C., & Bransby, D.P. (2023).  Psychological Safety Comes of Age: Observed Themes in an Established Literature. Annu. Rev. Organ. Psychol. Organ. Behav.,  10:55–78

Hewitt, D. B., Ellis, R. J., Hu, Y. Y., Cheung, E. O., Moskowitz, J. T., Agarwal, G., & Bilimoria, K. Y. (2020). Evaluating the association of multiple burnout definitions and thresholds with prevalence and outcomes. JAMA surgery, 155(11), 1043-1049.

Hughes, B. (2023). A Conceptual History of Psychology: The Mind Through Time. Bloomsbury Publishing.

Gallo, A. (2023). What is psychological safety? Harvard Business Review, Feb 15.

Goddiksen, M. P., Johansen, M. W., Armond, A. C., Clavien, C., Hogan, L., Kovács, N., ... & Lund, T. B. (2023). "The person in power told me to"—European PhD students' perspectives on guest authorship and good authorship practice. Plos one, 18(1), e0280018.

Grant, A. (2021). Think again: The power of knowing what you don't know. Penguin.

Khurana, R. (2010). From higher aims to hired hands: The social transformation of American business schools and the unfulfilled promise of management as a profession. Princeton University Press.

Lainidi, O., Jendeby, M. K., Montgomery, A., Mouratidis, C., Paitaridou, K., Cook, C., ... & Karakasidou, E. (2023). An integrative systematic review of employee silence and voice in healthcare: what are we really measuring?. Frontiers in Psychiatry, 14, 1111579.

Lesener, T., Gusy, B., & Wolter, C. (2019). The job demands-resources model: A meta-analytic review of longitudinal studies. Work & Stress, 33(1), 76-103.

Maslach, C., & Leiter, M. P. (2022). The burnout challenge: Managing people's relationships with their jobs. Harvard University Press.

Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological inquiry, 1(2), 108-141.

Montgomery, A. (2014). The inevitability of physician burnout: implications for interventions. Burnout Research, 1(1), 50-56.

Montgomery, A., Panagopoulou, E., Esmail, A., Richards, T., & Maslach, C. (2019). Burnout in healthcare: the case for organisational change. Bmj, 366.

Montgomery, A., & Lainidi, O. (2023). Creating healthy workplaces in healthcare: Are we delaying progress by focusing on what we can do rather than what we should do?. Frontiers in Public Health, 11, 1105009.

Montgomery, A., Lainidi, O., Johnson, J., Creese, J., Baathe, F., Baban, A., ... & Vohra, V. (2023). Employee silence in health care: Charting new avenues for leadership and management. Health Care Management Review, 48(1), 52-60

Paitaridou, K., Lainidi, O. & Montgomery, A. (2023, in press). Locating academic staff wellbeing within the organization: The interplay between personality, workload and burnout among academics. In M. Edwards, A. Martin & N. Ashkanasy (Eds), Handbook of Academic Mental Health..

Prasad, A. (2023). What's up with our obsession with the theoretical contribution: A means to an end or an end in and of itself?. Organization, 13505084231183079.

Ravishankar, R.A. (2022). A guide to Building Psychological Safety on Your Team.

Robinson DN. (1995). An intellectual history of psychology. Madison, WI: University of Wisconsin Press.

Teplitskiy, M., Duede, E., Menietti, M., & Lakhani, K. R. (2022). How status of research papers affects the way they are read and cited. Research Policy, 51(4), 104484.