Library / The Harm Done by Tests of Significance

Reference

Ezra Hauer “The harm done by tests of significance” (2004) // Accident Analysis & Prevention. Publisher: Elsevier BV. Vol. 36. No 3. Pp. 495–500. DOI: 10.1016/s0001-4575(03)00036-8

Abstract

Three historical episodes in which the application of null hypothesis significance testing (NHST) led to the mis-interpretation of data are described. It is argued that the pervasive use of this statistical ritual impedes the accumulation of knowledge and is unfit for use.

Bib

@Article{hauer2004,
  title = {The harm done by tests of significance},
  volume = {36},
  issn = {0001-4575},
  url = {http://dx.doi.org/10.1016/S0001-4575(03)00036-8},
  abstract = {Three historical episodes in which the application of null hypothesis significance testing (NHST) led to the mis-interpretation of data are described. It is argued that the pervasive use of this statistical ritual impedes the accumulation of knowledge and is unfit for use.},
  doi = {10.1016/s0001-4575(03)00036-8},
  number = {3},
  journal = {Accident Analysis & Prevention},
  publisher = {Elsevier BV},
  author = {Hauer, Ezra},
  year = {2004},
  month = {may},
  pages = {495–500}
}

Quotes (1)

The Importance of Effect Magnitude Over Statistical Hypotheses

Instead of asking “how many more crashes?” and the authors chose to ask “are we sufficiently sure that the effect was not zero?” This substitution of questions led to all the subsequent entanglements. These can all be avoided by not testing statistical hypotheses when the research question is about the effect of some treatment; by returning to common sense and the mainstream of science and providing estimates of effect magnitude and its standard error instead.