Twelve P-Value Misconceptions

Twelve P-Value Misconceptions:

1. If P =.05, the null hypothesis has only a 5% chance of being true.
2. A nonsignificant difference (eg, P > .05) means there is no difference between groups.
3. A statistically significant finding is clinically important.
4. Studies with P values on opposite sides of .05 are conflicting.
5. Studies with the same P value provide the same evidence against the null hypothesis.
6. P = .05 means that we have observed data that would occur only 5% of the time under the null hypothesis.
7. P = .05 and P <= .05 mean the same thing.
8. P values are properly written as inequalities (eg, “P < .02” when P = .015)
9. P = .05 means that if you reject the null hypothesis, the probability of a type I error is only 5%.
10. With a P = .05 threshold for significance, the chance of a type I error will be 5%.
11. You should use a one-sided P value when you don’t care about a result in one direction, or a difference in that direction is impossible.
12. A scientific conclusion or treatment policy should be based on whether or not the P value is significant.