A Cautionary Note on the Use of Error Bars

Excerpts

Climate studies often involve comparisons between estimates of some parameter derived from different observed and/or model-generated datasets. It is common practice to present estimates of two or more statistical quantities with error bars about each representing a confidence interval. If the error bars do not overlap, it is presumed that there is a statistically significant difference between them. In general, such a procedure is not valid and usually results in declaring statistical significance too infrequently. Simple examples that demonstrate the nature of this pitfall, along with some formulations, are presented. It is recommended that practitioners use standard hypothesis testing techniques that have been derived from statistical theory rather than the ad hoc approach involving error bars.

Reference

John R Lanzante “A Cautionary Note on the Use of Error Bars” (2005) DOI: 10.1175/jcli3499.1

@Article{lanzante2005,
  title = {A Cautionary Note on the Use of Error Bars},
  volume = {18},
  issn = {0894-8755},
  url = {http://dx.doi.org/10.1175/JCLI3499.1},
  doi = {10.1175/jcli3499.1},
  number = {17},
  journal = {Journal of Climate},
  publisher = {American Meteorological Society},
  author = {Lanzante, John R},
  year = {2005},
  month = {sep},
  pages = {3699–3703}
}