The Influence of Journal Prestige on the Inflation of Effect Sizes in Small Trials


We found that small trials published in NEJM, JAMA and Lancet were more likely to display more favourable results for experimental interventions compared with trials in other publication venues. Inflated effect sizes were seen primarily for early small trials in these prominent journals. Therefore, the results of small trials with spectacular early promises for large treatment effects should be seen with great caution. Conversely, for large trials, effect estimates are likely to be more reliable. Small-study effects have been previously documented in the randomized trials literature. However, the results of our study provide further insight suggesting the possibility of a specific interaction with further exaggerated effects when the limited evidence from small trials appears in the most prestigious journals. Also, the inflation of effects in early randomized trials on particular interventions appears to be quite specific to these most prestigious journals. Some modest heterogeneity was seen in the two tertiles with higher events, but heterogeneity is difficult to determine in the tertile with lower events, because of the wide uncertainty in the ROR for single topics, when there is limited evidence.