Magnitude of effects in clinical trials published in high-impact general medical journals

Excerpts

We found that small trials published in NEJM, JAMA and Lancet were more likely to display more favourable results for experimental interventions compared with trials in other publication venues. Inflated effect sizes were seen primarily for early small trials in these prominent journals. Therefore, the results of small trials with spectacular early promises for large treatment effects should be seen with great caution. Conversely, for large trials, effect estimates are likely to be more reliable. Small-study effects have been previously documented in the randomized trials literature. However, the results of our study provide further insight suggesting the possibility of a specific interaction with further exaggerated effects when the limited evidence from small trials appears in the most prestigious journals. Also, the inflation of effects in early randomized trials on particular interventions appears to be quite specific to these most prestigious journals. Some modest heterogeneity was seen in the two tertiles with higher events, but heterogeneity is difficult to determine in the tertile with lower events, because of the wide uncertainty in the ROR for single topics, when there is limited evidence.

Reference

Konstantinos CM Siontis, Evangelos Evangelou, John PA Ioannidis “Magnitude of effects in clinical trials published in high-impact general medical journals” (2011) DOI: 10.1093/ije/dyr095

@Article{siontis2011,
  title = {Magnitude of effects in clinical trials published in high-impact general medical journals},
  volume = {40},
  issn = {0300-5771},
  url = {http://dx.doi.org/10.1093/ije/dyr095},
  doi = {10.1093/ije/dyr095},
  number = {5},
  journal = {International Journal of Epidemiology},
  publisher = {Oxford University Press (OUP)},
  author = {Siontis, Konstantinos CM and Evangelou, Evangelos and Ioannidis, John PA},
  year = {2011},
  month = {sep},
  pages = {1280–1291}
}