Dicussed P-Value Beyond Any Usual Limit of Significance from berkson1938.
Reference
J. L. Hodges, E. L. Lehmann “Testing the Approximate Validity of Statistical Hypotheses” (1954) // Journal of the Royal Statistical Society: Series B (Methodological). Publisher: Wiley. Vol. 16. No 2. Pp. 261–268. DOI: 10.1111/j.2517-6161.1954.tb00169.x
Abstract
The distinction between statistical significance and material significance in hypotheses testing is discussed. Modifications of the customary tests, in order to test for the absence of material significance, are derived for several parametric problems, for the chi-square test of goodness of fit, and for Student’s hypothesis. The latter permits one to test the hypothesis that the means of two normal populations of equal variance, do not differ by more than a stated amount.
Bib
@Article{hodges1954,
title = {Testing the Approximate Validity of Statistical Hypotheses},
abstract = {The distinction between statistical significance and material significance in hypotheses testing is discussed. Modifications of the customary tests, in order to test for the absence of material significance, are derived for several parametric problems, for the chi-square test of goodness of fit, and for Student's hypothesis. The latter permits one to test the hypothesis that the means of two normal populations of equal variance, do not differ by more than a stated amount.},
volume = {16},
issn = {2517-6161},
doi = {10.1111/j.2517-6161.1954.tb00169.x},
number = {2},
journal = {Journal of the Royal Statistical Society: Series B (Methodological)},
publisher = {Wiley},
author = {Hodges, J. L. and Lehmann, E. L.},
year = {1954},
month = {jul},
pages = {261–268}
}
Quotes (2)
Natural Population is Ever Exactly Normal
For example, we may formulate the hypothesis that a population is normally distributed, but we realize that no natural population is ever exactly normal. We would want to reject normality only if the departure of the actual distribution from the normal form were great enough to be material for our investigation.
Material Significance
Page 261It seems to us that this difficulty can be avoided by making a clear distinction, in the formulation of the problem, between “statistical significance” and what might be called “material significance”.