I have just published a preprint of a paper ‘Quantile absolute deviation’. It’s based on a series of my research notes that I have been writing since December 2020.
The paper preprint is available on arXiv: arXiv:2208.13459 [stat.ME]. The paper source code is available on GitHub: AndreyAkinshin/paper-qad. You can cite it as follows:
- Andrey Akinshin (2022) “Quantile absolute deviation” arXiv:2208.13459
Abstract:
The median absolute deviation (MAD) is a popular robust measure of statistical dispersion. However, when it is applied to non-parametric distributions (especially multimodal, discrete, or heavy-tailed), lots of statistical inference issues arise. Even when it is applied to distributions with slight deviations from normality and these issues are not actual, the Gaussian efficiency of the MAD is only 37% which is not always enough.
In this paper, we introduce the quantile absolute deviation (QAD) as a generalization of the MAD. This measure of dispersion provides a flexible approach to analyzing properties of non-parametric distributions. It also allows controlling the trade-off between robustness and statistical efficiency. We use the trimmed Harrell-Davis median estimator based on the highest density interval of the given width as a complimentary median estimator that gives increased finite-sample Gaussian efficiency compared to the sample median and a breakdown point matched to the QAD.
As a rule of thumb, we suggest using two new measures of dispersion called the standard QAD and the optimal QAD. They give 54% and 65% of Gaussian efficiency having breakdown points of 32% and 14% respectively.
Relevant blog posts
Here is the full list of the relevant blog posts:
- Quantile absolute deviation: estimating statistical dispersion around quantiles (December 1, 2020)
- Middle non-zero quantile absolute deviation (February 15, 2022)
- Gamma effect size powered by the middle non-zero quantile absolute deviation (February 22, 2022)
- Middle non-zero quantile absolute deviation, Part 2 (June 28, 2022)
- Untied quantile absolute deviation (July 5, 2022)
- Degenerate point of dispersion estimators (July 12, 2022)
- Caveats of using the median absolute deviation (August 2, 2022)
- Asymptotic Gaussian efficiency of the quantile absolute deviation (August 16, 2022)
- Standard quantile absolute deviation (August 23, 2022)
- Quantile absolute deviation of the Normal distribution (August 24, 2022)
- Quantile absolute deviation of the Uniform distribution (August 25, 2022)
- Quantile absolute deviation of the Exponential distribution (August 26, 2022)
- Quantile absolute deviation of the Pareto distribution (August 29, 2022)
- Optimal quantile absolute deviation (August 30, 2022)
- Standard trimmed Harrell-Davis median estimator (August 31, 2022)
- Preprint announcement: 'Quantile absolute deviation' (September 1, 2022)
BibTeX reference
@article{akinshin2022qad,
title = {Quantile absolute deviation},
author = {Akinshin, Andrey},
year = {2022},
month = {8},
publisher = {arXiv},
doi = {10.48550/ARXIV.2208.13459},
url = {https://arxiv.org/abs/2208.13459}
}