Excerpts / Twelve P-Value Misconceptions

Twelve P-Value Misconceptions:

  1. If P =.05, the null hypothesis has only a 5% chance of being true.
  2. A nonsignificant difference (eg, P > .05) means there is no difference between groups.
  3. A statistically significant finding is clinically important.
  4. Studies with P values on opposite sides of .05 are conflicting.
  5. Studies with the same P value provide the same evidence against the null hypothesis.
  6. P = .05 means that we have observed data that would occur only 5% of the time under the null hypothesis.
  7. P = .05 and P <= .05 mean the same thing.
  8. P values are properly written as inequalities (eg, “P < .02” when P = .015)
  9. P = .05 means that if you reject the null hypothesis, the probability of a type I error is only 5%.
  10. With a P = .05 threshold for significance, the chance of a type I error will be 5%.
  11. You should use a one-sided P value when you don’t care about a result in one direction, or a difference in that direction is impossible.
  12. A scientific conclusion or treatment policy should be based on whether or not the P value is significant.