The Wrong Question

Statisticians classically asked the wrong question — and were willing to answer with a lie, one that was often a downright lie. They asked “Are the effects of A and B different?” and they were willing to answer “no.”

All we know about the world teaches us that the effects of A and B are always difference — in some decimal place — for any A and B. Thus asking “Are the effects different?” is foolish.

What we should be answering first is “Can we tell the direction in which the effects of A differ from the effects of B?” In other words, can we be confident about the direction from A to B? Is it “up,” “down” or “uncertain”?

The third answer to this first question is that we are “uncertain about the direction” — it is not, and never should be, that we “accept the null hypothesis.”