Cycle 90: c=2 capped AD sum vs plain Anderson-Darling vs Shapiro-Wilk (seed 90101)

Script: stuff/s2-vs-ad-vs-sw-seed90101.py (run with uv run --with numpy --with scipy python ..., 26 s).

Definitions (rule 1)

For a sample x_1..x_n, let z_(1) <= ... <= z_(n) be the order statistics of the sample-standardized values (x - mean)/sd, sd with n-1. The Anderson-Darling term is

t_i = -((2i-1)/n) * [ ln Phi(z_(i)) + ln(1 - Phi(z_(n+1-i))) ], i = 1..n.

The three competing statistics:

• S2 (capped AD sum, c=2): S2 = sum_i log(1 + 2 t_i) / 2. Reject for large S2.
• A2 (plain Anderson-Darling): A2 = sum_i t_i - n. Reject for large A2.
• W (Shapiro-Wilk, scipy.stats.shapiro): reject for small W.

Each statistic gets its own normal-calibrated 5% critical value: the 95th percentile of S2 and A2 and the 5th percentile of W over the same 20000 N(0,1) samples per n. All three tests are then applied to the same simulated samples under every law, so differences are paired. The paired Monte Carlo SE is sd(d_j)/sqrt(20000) where d_j is the difference of the per-sample 0/1 rejection indicators.

Seed 90101 (numpy default_rng), 20000 sims per (n, law), n in 25, 50, 100. Laws: normal, t(3), t(5), 0.9 N(0,1) + 0.1 N(0,9) (per-observation mixture), chi-square(5).

Results

"win" = S2 minus competitor exceeds +2 SE; "tie" = within 2 SE; "loss" = below -2 SE. Normal rows are size checks: all three tests sit at exactly 0.0500 by construction (critical values are quantiles of these same samples), so the difference is 0.

Classification per n and law (rule 2)

• t(3): S2 beats A2 and beats W at n=25, 50, 100 (41, 50, 44 SE over A2; 34, 40, 34 SE over W).
• t(5): S2 beats A2 and beats W at n=25, 50, 100 (35, 52, 70 SE over A2; 24, 36, 45 SE over W).
• 0.9 N(0,1) + 0.1 N(0,9): S2 beats A2 and beats W at n=25, 50, 100 (42, 55, 61 SE over A2; 23, 23, 19 SE over W).
• chi-square(5): S2 ties A2 at n=25 (+2.0 SE, inside the 2 SE band), loses to A2 at n=50 (-31 SE) and n=100 (-33 SE); S2 loses to W at every n (-36, -61, -37 SE).

Overall verdict

S2 loses most to A2 at chi-square(5), n=100 (-0.055, -33 SE; n=50 is nearly as bad at -0.077, -31 SE), and loses most to W at chi-square(5), n=50 (-0.160, -61 SE). It loses nowhere else: on every symmetric heavy-tailed or contaminated law it wins over both competitors by 19 to 70 SE. The cap at c=2 buys its heavy-tail power by damping the few extreme t_i that a one-sided skewed tail produces, which is exactly the signal A2 and W use to detect chi-square(5).

One sentence: The c=2 capped AD sum is worth using over plain AD and over Shapiro-Wilk only when the alternative is symmetric heavy tails or contamination, where it wins by 4 to 20 points of power at every n; it is not worth using as a general-purpose normality test, because under the skewed chi-square(5) law it loses 5 to 8 points to plain AD at n >= 50 and 7 to 16 points to Shapiro-Wilk at every n.

Cycle 90 replication: c=2 capped AD sum vs plain Anderson-Darling vs Shapiro-Wilk (seed 90303)

Script: stuff/s2-vs-ad-vs-sw-seed90303.py (run with uv run --with numpy --with scipy python ..., 26 s).

Definitions (rule 1)

For a sample x_1..x_n, let z_(1) <= ... <= z_(n) be the order statistics of the sample-standardized values (x - mean)/sd, sd with n-1. The Anderson-Darling term is

t_i = -((2i-1)/n) * [ ln Phi(z_(i)) + ln(1 - Phi(z_(n+1-i))) ], i = 1..n.

The three competing statistics:

• S2 (capped AD sum, c=2): S2 = sum_i log(1 + 2 t_i) / 2. Reject for large S2.
• A2 (plain Anderson-Darling): A2 = sum_i t_i - n. Reject for large A2.
• W (Shapiro-Wilk, scipy.stats.shapiro): reject for small W.

Each statistic gets its own normal-calibrated 5% critical value: the 95th percentile of S2 and A2 and the 5th percentile of W over the same 20000 N(0,1) samples per n. All three tests are then applied to the same simulated samples under every law, so differences are paired. The paired Monte Carlo SE is sd(d_j)/sqrt(20000) where d_j is the difference of the per-sample 0/1 rejection indicators.

Seed 90303 (numpy default_rng), 20000 sims per (n, law), n in 25, 50, 100. Laws: normal, t(3), t(5), 0.9 N(0,1) + 0.1 N(0,9) (per-observation mixture), chi-square(5).

Results

Calls: win = S2 ahead by more than two paired SE, tie = within two SE, loss = behind by more than two SE; "size" rows are the calibration sample.

Per-cell verdicts (rule 2)

• t(3), t(5), contaminated normal: S2 beats A2 at every n (+0.061 to +0.193, 28 to 67 SE) and beats W at every n (+0.034 to +0.117, 18 to 45 SE).
• chi-square(5) vs A2: loss at n=25 (-0.0063, -2.5 SE, only just past the two-SE line; seed 90101 called this cell a tie), loss at n=50 (-0.0825, -33.0 SE) and n=100 (-0.0573, -34.1 SE).
• chi-square(5) vs W: loss at every n: n=25 -0.0862 (-41 SE), n=50 -0.1628 (-62.3 SE), n=100 -0.0697 (-38.7 SE).

Overall verdict

S2 loses anywhere only under chi-square(5). Against A2 the largest drop in power is at n=50 (-0.083), the largest in SE units at n=100 (-34.1 SE). Against W the worst cell is chi-square(5) at n=50 (-0.163, -62.3 SE). This matches seed 90101 cell for cell except chi-square(5) n=25 vs A2, which moves from tie to a marginal loss, so the sign of that cell is not settled and its size is under 0.01 either way.

Sentence for the heap-death learning: The c=2 capped AD sum is worth using over both plain AD and Shapiro-Wilk for symmetric heavy-tailed or scale-contaminated alternatives, where it gains 3 to 19 points of power at n=25-100, but it is not worth using as a general omnibus test because it loses to plain AD by up to 8 points and to Shapiro-Wilk by up to 16 points under chi-square(5).

Cycle 90 third replication: c=2 capped AD sum vs plain Anderson-Darling vs Shapiro-Wilk (seed 90505)

Script: stuff/s2-vs-ad-vs-sw-seed90505.py (run with uv run --with numpy --with scipy python ..., 26 s). Same code as the seed 90303 script with only the seed changed.

Definitions (rule 1)

For a sample x_1..x_n, let z_(1) <= ... <= z_(n) be the order statistics of the sample-standardized values (x - mean)/sd, sd with n-1. The Anderson-Darling term is

t_i = -((2i-1)/n) * [ ln Phi(z_(i)) + ln(1 - Phi(z_(n+1-i))) ], i = 1..n.

The three competing statistics:

• S2 (capped AD sum, c=2): S2 = sum_i log(1 + 2 t_i) / 2. Reject for large S2.
• A2 (plain Anderson-Darling): A2 = sum_i t_i - n. Reject for large A2.
• W (Shapiro-Wilk, scipy.stats.shapiro): reject for small W.

Each statistic gets its own normal-calibrated 5% critical value: the 95th percentile of S2 and A2 and the 5th percentile of W over the same 20000 N(0,1) samples per n. All three tests are then applied to the same simulated samples under every law, so differences are paired. The paired Monte Carlo SE is sd(d_j)/sqrt(20000) where d_j is the difference of the per-sample 0/1 rejection indicators.

Seed 90505 (numpy default_rng), 20000 sims per (n, law), n in 25, 50, 100. Laws: normal, t(3), t(5), 0.9 N(0,1) + 0.1 N(0,9) (per-observation mixture), chi-square(5).

Results

Calls: win = S2 ahead by more than two paired SE, tie = within two SE, loss = behind by more than two SE; "size" rows are the calibration sample.

Per-cell verdicts (rule 2)

• t(3), t(5), contaminated normal: S2 beats A2 at every n (+0.072 to +0.201, 33 to 69 SE) and beats W at every n (+0.032 to +0.117, 18 to 45 SE).
• chi-square(5) vs A2: tie at n=25 (-0.0003, -0.1 SE), loss at n=50 (-0.0781, -31.2 SE) and n=100 (-0.0577, -34.2 SE).
• chi-square(5) vs W: loss at every n: n=25 -0.0842 (-40 SE), n=50 -0.1642 (-62.5 SE), n=100 -0.0699 (-38.7 SE).

Overall verdict

S2 loses anywhere only under chi-square(5). Against A2 the largest drop in power is at n=50 (-0.078), the largest in SE units at n=100 (-34.2 SE). Against W the worst cell is chi-square(5) at n=50 (-0.164, -62.5 SE).

Across the three seeds (90101, 90303, 90505) every cell has the same call except chi-square(5) n=25 vs A2, which reads tie, loss (-2.5 SE), tie. Its difference is between -0.006 and 0.000 in all three runs, so that cell is a genuine tie at the resolution of 20000 sims: S2 and A2 have the same chi-square(5) power at n=25 and S2 falls behind only from n=50 on.

Sentence for the heap-death learning: The c=2 capped AD sum is worth using over both plain AD and Shapiro-Wilk for symmetric heavy-tailed or scale-contaminated alternatives, where it gains 3 to 20 points of power at n=25-100 in all three seeds, but it is not worth using as a general omnibus test because under chi-square(5) it loses to plain AD by up to 8 points and to Shapiro-Wilk by up to 16 points.