Rebuttal - Special Collection: Open Science Practices - a vision for the future of SAJIP

Reducing our dependence on null hypothesis testing: A key to enhance the reproducibility and credibility of our science

Kevin R. Murphy
SA Journal of Industrial Psychology | Vol 45 | a1717 | DOI: https://doi.org/10.4102/sajip.v45i0.1717 | © 2019 Kevin R. Murphy | This work is licensed under CC Attribution 4.0
Submitted: 16 July 2019 | Published: 05 November 2019

About the author(s)

Kevin R. Murphy, Department of Work and Employment Studies, Kemmy Business School, University of Limerick, Limerick, Ireland

Abstract

Problemification: Over-reliance on null hypothesis significance testing (NHST) is one of the most important causes of the emerging crisis over the credibility and reproducibility of our science.

Implications: Most studies in the behavioural and social sciences have low levels of statistical power. Because ‘significant’ results are often required, but often difficult to produce, the temptation to engage in questionable research practices that will produce these results is immense.

Purpose: Methodologists have been trying for decades to convince researchers, reviewers and editors that significance tests are neither informative nor useful. A recent set of articles published in top journals and endorsed by hundreds of scientists around the world seem to provide a fresh impetus for overturning the practice of using NHST as the primary, and sometimes sole basis for evaluating research results.

Recommendations: Authors, reviewers and journal editors are asked to change long-engrained habits and realise that ‘statistically significant’ says more about the design of one’s study than about the importance of one’s results. They are urged to embrace the ATOM principle in evaluating research results, that is, accept that there will always be uncertainty, and be thoughtful, open and modest in evaluating what the data mean.


Keywords

Significance Testing; Confidence Intervals; Questionable Research Practices; Null Hypothesis

Metrics

Total abstract views: 2239
Total article views: 2902


Crossref Citations

No related citations found.