Of beauty, sex, and power: statistical challenges in estimating small effects

Excerpts

This is a Type M (magnitude) error (Gelman and Tuerlinckx, 2000): the study is constructed in such a way that any statistically-significant finding will almost certainly be a huge overestimate of the true effect. In addition there will be Type S (sign) errors, in which the estimate will be in the opposite direction as the true effect.

Reference

A Gelman, D Weakliem “Of beauty, sex, and power: statistical challenges in estimating small effects” (2009) DOI: 10.1511/2009.79.310

@Article{gelman2009,
  title = {Of beauty, sex, and power: statistical challenges in estimating small effects},
  volume = {97},
  issn = {1545-2786},
  url = {http://dx.doi.org/10.1511/2009.79.310},
  doi = {10.1511/2009.79.310},
  number = {4},
  journal = {American Scientist},
  publisher = {Sigma Xi},
  author = {A Gelman and D Weakliem},
  year = {2009},
  pages = {310},
  custom-url-pdf = {http://www.stat.columbia.edu/~gelman/research/unpublished/power.pdf}
}