In the previous post, I suggested an asymmetric decile-based outlier detector as an alternative to Tukey’s fences. In this post, we run some numerical simulations to check out the suggested outlier detector in action.
Let $Q_p$ be an estimation of the $p^\textrm{th}$ quantile for the given sample, and $k$ be a parameter of the outlier detector. With this notation, we consider the following range:
$$ [Q_{0.1} - k (Q_{0.5} - Q_{0.1}),\, Q_{0.9} + k (Q_{0.9} - Q_{0.5})] $$All the sample elements outside this range should be considered as outliers.
Now let’s repeat the experiment that I performed for Tukey’s fences according to the following scheme:
- Enumerate different distributions. We consider the standard normal distribution, the standard Gumbel distribution, and the standard exponential distribution.
- Enumerate different $k$ values. We consider $k \in \{ 1.0, 1.5, 2.0, 2.5, 3.0, 3.5 \}$.
- Enumerate different samples sizes from 6 to 500.
- Generate 1000 random samples from the given distribution of the given size.
- For each sample, detect outliers using the considered $k$ values.
- Evaluate the percentage of samples that contain at least one outlier.
Here are the results:
