Cycle 91: S2+A2 combinations (Bonferroni max-p, reject-if-either) vs plain AD vs Shapiro-Wilk (seed 91202)

Script: stuff/s2a2-combo-vs-ad-vs-sw-seed91202.py (run with uv run --with numpy --with scipy python ..., 25 s). Seed 91202 (numpy default_rng) was not used in cycles 88-90.

Definitions (before any table, per rule 1)

For a sample x_1..x_n, z = (x - mean)/sd with sd on n-1, z_(i) sorted. The Anderson-Darling term is

t_i = -((2i-1)/n) * [ ln Phi(z_(i)) + ln(1 - Phi(z_(n+1-i))) ], i = 1..n.

• S2 = sum_i log(1 + 2 t_i) / 2 (capped AD sum, c=2). Reject for large S2.
• A2 = sum_i t_i - n (plain Anderson-Darling). Reject for large A2.
• W = Shapiro-Wilk (scipy.stats.shapiro). Reject for small W.
• Normal calibration: for each n, 20000 N(0,1) samples give the null distribution of S2, A2, W. p_S2 and p_A2 are upper-tail empirical p-values against that null sample.
• bonf-maxp (Bonferroni max-p): reject when min(p_S2, p_A2) < 0.025.
• either-2.5 (reject-if-either): reject when S2 exceeds its normal-calibrated 97.5% quantile or A2 exceeds its normal-calibrated 97.5% quantile.
• Comparators A2, W and S2-alone each use their own normal-calibrated 5% critical value.
• Paired difference = mean over the same 20000 samples of (combo rejects) - (comparator rejects); SE = sd of that per-sample difference / sqrt(20000). win/loss = outside two SE; tie = inside. vs S2 reports whether the combination stays within two SE of S2 alone (within / above / below).

Results (20000 sims per (n, law), seed 91202)

Losing cells (combination loses by more than two paired SE):

• bonf-maxp, n=25, t(3), vs W: diff -0.0151 (-10.0 SE)
• either-2.5, n=25, t(3), vs W: diff -0.0147 (-9.8 SE)
• bonf-maxp, n=25, t(5), vs W: diff -0.0208 (-14.0 SE)
• either-2.5, n=25, t(5), vs W: diff -0.0204 (-13.8 SE)
• bonf-maxp, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0302 (-19.6 SE)
• either-2.5, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0300 (-19.5 SE)
• bonf-maxp, n=25, chi2(5), vs A2: diff -0.0393 (-19.8 SE)
• bonf-maxp, n=25, chi2(5), vs W: diff -0.1216 (-52.5 SE)
• either-2.5, n=25, chi2(5), vs A2: diff -0.0387 (-19.5 SE)
• either-2.5, n=25, chi2(5), vs W: diff -0.1211 (-52.4 SE)
• bonf-maxp, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0131 (-7.0 SE)
• either-2.5, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0129 (-6.9 SE)
• bonf-maxp, n=50, chi2(5), vs A2: diff -0.0503 (-27.3 SE)
• bonf-maxp, n=50, chi2(5), vs W: diff -0.1360 (-56.1 SE)
• either-2.5, n=50, chi2(5), vs A2: diff -0.0500 (-27.2 SE)
• either-2.5, n=50, chi2(5), vs W: diff -0.1358 (-56.0 SE)
• bonf-maxp, n=100, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0042 (-2.4 SE)
• either-2.5, n=100, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0040 (-2.3 SE)
• bonf-maxp, n=100, chi2(5), vs A2: diff -0.0118 (-15.0 SE)
• bonf-maxp, n=100, chi2(5), vs W: diff -0.0222 (-21.3 SE)
• either-2.5, n=100, chi2(5), vs A2: diff -0.0117 (-15.0 SE)
• either-2.5, n=100, chi2(5), vs W: diff -0.0221 (-21.3 SE)

Per-rule-2 statement, per n and per combination

The two combinations are numerically almost identical in every cell (they differ by at most 0.0006 in power), which is expected: both reject when either statistic clears its own 2.5% point, the only difference being discrete p-value ties.

Verdict

Not omnibus-safe. Neither combination turns the chi-square(5) loss into a tie or a gain: both lose to plain AD at every n (-20, -27, -15 SE for n=25, 50, 100) and to Shapiro-Wilk at every n (-52, -56, -21 SE). They also give back a large part of S2's heavy-tail and contamination gains: every heavy-tail and contamination cell sits 22-40 SE below S2 alone, and at n=25 the combinations lose to Shapiro-Wilk on all three of those laws, which S2 alone did not.

Why: S2 and A2 are built from the same t_i and are highly correlated under the null, so the union of two 2.5% regions has actual size only 3.5-4.0% (see the normal rows). The combination spends level without buying independent evidence; on chi2(5) it sits between S2 and A2 but closer to S2, and on heavy tails it sits between S2 and A2 but well below S2.

Named losing cells (combination loses by more than two paired SE):

• bonf-maxp, n=25, t(3), vs W: diff -0.0151 (-10.0 SE)
• either-2.5, n=25, t(3), vs W: diff -0.0147 (-9.8 SE)
• bonf-maxp, n=25, t(5), vs W: diff -0.0208 (-14.0 SE)
• either-2.5, n=25, t(5), vs W: diff -0.0204 (-13.8 SE)
• bonf-maxp, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0302 (-19.6 SE)
• either-2.5, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0300 (-19.5 SE)
• bonf-maxp, n=25, chi2(5), vs A2: diff -0.0393 (-19.8 SE)
• bonf-maxp, n=25, chi2(5), vs W: diff -0.1216 (-52.5 SE)
• either-2.5, n=25, chi2(5), vs A2: diff -0.0387 (-19.5 SE)
• either-2.5, n=25, chi2(5), vs W: diff -0.1211 (-52.4 SE)
• bonf-maxp, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0131 (-7.0 SE)
• either-2.5, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0129 (-6.9 SE)
• bonf-maxp, n=50, chi2(5), vs A2: diff -0.0503 (-27.3 SE)
• bonf-maxp, n=50, chi2(5), vs W: diff -0.1360 (-56.1 SE)
• either-2.5, n=50, chi2(5), vs A2: diff -0.0500 (-27.2 SE)
• either-2.5, n=50, chi2(5), vs W: diff -0.1358 (-56.0 SE)
• bonf-maxp, n=100, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0042 (-2.4 SE)
• either-2.5, n=100, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0040 (-2.3 SE)
• bonf-maxp, n=100, chi2(5), vs A2: diff -0.0118 (-15.0 SE)
• bonf-maxp, n=100, chi2(5), vs W: diff -0.0222 (-21.3 SE)
• either-2.5, n=100, chi2(5), vs A2: diff -0.0117 (-15.0 SE)
• either-2.5, n=100, chi2(5), vs W: diff -0.0221 (-21.3 SE)

Not in the list (wins or ties): all heavy-tail and contamination cells vs A2 at every n; t(3) and t(5) vs W at n=50 and n=100.

Next step this suggests: pair S2 with a genuinely skew-sensitive statistic that is weakly correlated with it under the null (sample skewness b1, or Shapiro-Wilk itself) rather than with A2, and recover the lost level by calibrating the combined rule's 5% point jointly instead of by Bonferroni.

Cycle 91: S2+A2 combinations (Bonferroni max-p, reject-if-either) vs plain AD vs Shapiro-Wilk, replication at seed 91404

Script: stuff/s2a2-combo-vs-ad-vs-sw-seed91404.py (run with uv run --with numpy --with scipy python ..., 25 s). Seed 91404 (numpy default_rng) was not used in cycles 88-90, and differs from member-2's seed 91202. Same estimator and design as member-2's item; this is an independent-seed replication so that each verdict below can be checked against the 91202 run cell by cell.

Definitions (before any table, per rule 1)

For a sample x_1..x_n, z = (x - mean)/sd with sd on n-1, z_(i) sorted. The Anderson-Darling term is

t_i = -((2i-1)/n) * [ ln Phi(z_(i)) + ln(1 - Phi(z_(n+1-i))) ], i = 1..n.

• S2 = sum_i log(1 + 2 t_i) / 2 (capped AD sum, c=2). Reject for large S2.
• A2 = sum_i t_i - n (plain Anderson-Darling). Reject for large A2.
• W = Shapiro-Wilk (scipy.stats.shapiro). Reject for small W.
• Normal calibration: for each n, 20000 N(0,1) samples give the null distribution of S2, A2, W. p_S2 and p_A2 are upper-tail empirical p-values against that null sample.
• bonf-maxp (Bonferroni max-p): reject when min(p_S2, p_A2) < 0.025.
• either-2.5 (reject-if-either): reject when S2 exceeds its normal-calibrated 97.5% quantile or A2 exceeds its normal-calibrated 97.5% quantile.
• Comparators A2, W and S2-alone each use their own normal-calibrated 5% critical value.
• Paired difference = mean over the same 20000 samples of (combo rejects) - (comparator rejects); SE = sd of that per-sample difference / sqrt(20000). win/loss = outside two SE; tie = inside. vs S2 reports whether the combination stays within two SE of S2 alone (within / above / below).

Results (20000 sims per (n, law), seed 91404)

Losing cells (combination loses by more than two paired SE):

• bonf-maxp, n=25, t(3), vs W: diff -0.0106 (-6.9 SE)
• either-2.5, n=25, t(3), vs W: diff -0.0106 (-6.9 SE)
• bonf-maxp, n=25, t(5), vs W: diff -0.0204 (-13.9 SE)
• either-2.5, n=25, t(5), vs W: diff -0.0204 (-13.9 SE)
• bonf-maxp, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0255 (-16.9 SE)
• either-2.5, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0255 (-16.9 SE)
• bonf-maxp, n=25, chi2(5), vs A2: diff -0.0448 (-22.0 SE)
• bonf-maxp, n=25, chi2(5), vs W: diff -0.1197 (-52.0 SE)
• either-2.5, n=25, chi2(5), vs A2: diff -0.0447 (-22.0 SE)
• either-2.5, n=25, chi2(5), vs W: diff -0.1196 (-51.9 SE)
• bonf-maxp, n=50, t(5), vs W: diff -0.0068 (-3.6 SE)
• either-2.5, n=50, t(5), vs W: diff -0.0062 (-3.3 SE)
• bonf-maxp, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0221 (-12.0 SE)
• either-2.5, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0217 (-11.8 SE)
• bonf-maxp, n=50, chi2(5), vs A2: diff -0.0444 (-25.3 SE)
• bonf-maxp, n=50, chi2(5), vs W: diff -0.1356 (-55.8 SE)
• either-2.5, n=50, chi2(5), vs A2: diff -0.0441 (-25.1 SE)
• either-2.5, n=50, chi2(5), vs W: diff -0.1353 (-55.8 SE)
• bonf-maxp, n=100, chi2(5), vs A2: diff -0.0088 (-12.2 SE)
• bonf-maxp, n=100, chi2(5), vs W: diff -0.0189 (-19.6 SE)
• either-2.5, n=100, chi2(5), vs A2: diff -0.0086 (-12.1 SE)
• either-2.5, n=100, chi2(5), vs W: diff -0.0188 (-19.6 SE)

Per-rule-2 statement, per n and per combination

The two combinations are numerically identical to within 0.0007 in power in every cell, as at seed 91202, so one line per n covers both bonf-maxp and either-2.5.

Agreement with seed 91202: every chi2(5) cell is a loss under both seeds; every heavy-tail and contamination cell vs A2 is a win under both seeds except t(5) n=25, which is a tie here and a win there (borderline); vs W the two seeds agree in all cells except t(5) n=50 (loss here at -3.6 SE, tie there) and contamination n=100 (tie here, loss at -2.4 SE there). Those three cells are the only seed-sensitive ones and all sit near the two-SE line; nothing changes sign between seeds.

Verdict

Not omnibus-safe. Replicated at a second seed: neither combination turns the chi-square(5) loss into a tie or a gain. Both lose to plain AD at every n (-22, -25, -12 SE for n=25, 50, 100) and to Shapiro-Wilk at every n (-52, -56, -20 SE). Both give back most of S2's heavy-tail and contamination edge: every such cell is 24-44 SE below S2 alone, and at n=25 both combinations lose to Shapiro-Wilk on all three of those laws.

Mechanism, as at seed 91202: S2 and A2 share the same t_i and are strongly correlated under the null, so the union of two 2.5% regions has actual size 3.5-4.0% (normal rows). The rule pays level without gaining an independent view of skew.

Named losing cells (combination loses by more than two paired SE):

• bonf-maxp, n=25, t(3), vs W: diff -0.0106 (-6.9 SE)
• either-2.5, n=25, t(3), vs W: diff -0.0106 (-6.9 SE)
• bonf-maxp, n=25, t(5), vs W: diff -0.0204 (-13.9 SE)
• either-2.5, n=25, t(5), vs W: diff -0.0204 (-13.9 SE)
• bonf-maxp, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0255 (-16.9 SE)
• either-2.5, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0255 (-16.9 SE)
• bonf-maxp, n=25, chi2(5), vs A2: diff -0.0448 (-22.0 SE)
• bonf-maxp, n=25, chi2(5), vs W: diff -0.1197 (-52.0 SE)
• either-2.5, n=25, chi2(5), vs A2: diff -0.0447 (-22.0 SE)
• either-2.5, n=25, chi2(5), vs W: diff -0.1196 (-51.9 SE)
• bonf-maxp, n=50, t(5), vs W: diff -0.0068 (-3.6 SE)
• either-2.5, n=50, t(5), vs W: diff -0.0062 (-3.3 SE)
• bonf-maxp, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0221 (-12.0 SE)
• either-2.5, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0217 (-11.8 SE)
• bonf-maxp, n=50, chi2(5), vs A2: diff -0.0444 (-25.3 SE)
• bonf-maxp, n=50, chi2(5), vs W: diff -0.1356 (-55.8 SE)
• either-2.5, n=50, chi2(5), vs A2: diff -0.0441 (-25.1 SE)
• either-2.5, n=50, chi2(5), vs W: diff -0.1353 (-55.8 SE)
• bonf-maxp, n=100, chi2(5), vs A2: diff -0.0088 (-12.2 SE)
• bonf-maxp, n=100, chi2(5), vs W: diff -0.0189 (-19.6 SE)
• either-2.5, n=100, chi2(5), vs A2: diff -0.0086 (-12.1 SE)
• either-2.5, n=100, chi2(5), vs W: diff -0.0188 (-19.6 SE)

Not in the list (wins or ties): all heavy-tail and contamination cells vs A2 at every n; t(3) vs W at n=50 and 100; t(5) vs W at n=100; contamination vs W at n=100.

Next step this suggests, same as member-2's: replace A2 with a partner that is weakly correlated with S2 under the null and sees skew directly (sample skewness sqrt(b1), or Shapiro-Wilk itself), and calibrate the combined 5% point jointly rather than by Bonferroni so the level is not wasted.

Cycle 91: S2+A2 combinations (Bonferroni max-p, reject-if-either) vs plain AD vs Shapiro-Wilk, third seed 91606, plus a size-calibrated min-p check

Script: stuff/s2a2-combo-vs-ad-vs-sw-seed91606.py (run with uv run --with numpy --with scipy python ..., 27 s). Seed 91606 (numpy default_rng) was not used in cycles 88-90 and differs from the cycle 91 seeds 91202 and 91404. Same estimator and design as the two earlier items. Motivation for the extra row: both earlier items show the combinations rejecting only 3.5-4.0% of normal samples, not 5%, because S2 and A2 are strongly correlated and Bonferroni does not credit that. This item adds minp-cal, the same min-p statistic calibrated to exact 5% size, to separate "the combination loses on chi2(5) because it is conservative" from "the combination loses because S2 drags A2 down". Rule 1 and rule 2 rows (bonf-maxp, either-2.5) are all present and judged on their own.

Definitions (before any table, per rule 1)

For a sample x_1..x_n, z = (x - mean)/sd with sd on n-1, z_(i) sorted. The Anderson-Darling term is

t_i = -((2i-1)/n) * [ ln Phi(z_(i)) + ln(1 - Phi(z_(n+1-i))) ], i = 1..n.

• S2 = sum_i log(1 + 2 t_i) / 2 (capped AD sum, c=2). Reject for large S2.
• A2 = sum_i t_i - n (plain Anderson-Darling). Reject for large A2.
• W = Shapiro-Wilk (scipy.stats.shapiro). Reject for small W.
• Normal calibration: for each n, 20000 N(0,1) samples give the null distribution of S2, A2, W. p_S2 and p_A2 are upper-tail empirical p-values against that null sample.
• bonf-maxp (Bonferroni max-p): reject when min(p_S2, p_A2) < 0.025.
• either-2.5 (reject-if-either): reject when S2 exceeds its normal-calibrated 97.5% quantile or A2 exceeds its normal-calibrated 97.5% quantile.
• minp-cal (extra, not required by the rules): reject when min(p_S2, p_A2) is below the normal-calibrated 5% quantile of min(p_S2, p_A2) itself, so its size is 5% by construction on the calibration sample.
• Comparators A2, W and S2-alone each use their own normal-calibrated 5% critical value.
• Paired difference = mean over the same 20000 samples of (combo rejects) - (comparator rejects); SE = sd of that per-sample difference / sqrt(20000). win/loss = outside two SE; tie = inside. vs S2 reports whether the combination stays within two SE of S2 alone (within / above / below).

Results (20000 sims per (n, law), seed 91606)

Losing cells (combination loses by more than two paired SE), as printed by the script

• bonf-maxp, n=25, t(3), vs W: diff -0.0116 (-7.5 SE)
• either-2.5, n=25, t(3), vs W: diff -0.0115 (-7.4 SE)
• bonf-maxp, n=25, t(5), vs W: diff -0.0213 (-14.5 SE)
• either-2.5, n=25, t(5), vs W: diff -0.0211 (-14.4 SE)
• bonf-maxp, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0259 (-16.9 SE)
• either-2.5, n=25, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0256 (-16.8 SE)
• bonf-maxp, n=25, chi2(5), vs A2: diff -0.0389 (-19.4 SE)
• bonf-maxp, n=25, chi2(5), vs W: diff -0.1153 (-50.7 SE)
• either-2.5, n=25, chi2(5), vs A2: diff -0.0387 (-19.3 SE)
• either-2.5, n=25, chi2(5), vs W: diff -0.1152 (-50.7 SE)
• minp-cal, n=25, chi2(5), vs W: diff -0.0660 (-35.5 SE)
• bonf-maxp, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0205 (-11.1 SE)
• either-2.5, n=50, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0204 (-11.1 SE)
• bonf-maxp, n=50, chi2(5), vs A2: diff -0.0500 (-26.9 SE)
• bonf-maxp, n=50, chi2(5), vs W: diff -0.1363 (-56.1 SE)
• either-2.5, n=50, chi2(5), vs A2: diff -0.0498 (-26.8 SE)
• either-2.5, n=50, chi2(5), vs W: diff -0.1361 (-56.1 SE)
• minp-cal, n=50, chi2(5), vs A2: diff -0.0173 (-10.9 SE)
• minp-cal, n=50, chi2(5), vs W: diff -0.1035 (-47.8 SE)
• bonf-maxp, n=100, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0081 (-4.6 SE)
• either-2.5, n=100, 0.9N(0,1)+0.1N(0,9), vs W: diff -0.0073 (-4.2 SE)
• bonf-maxp, n=100, chi2(5), vs A2: diff -0.0109 (-14.5 SE)
• bonf-maxp, n=100, chi2(5), vs W: diff -0.0203 (-20.4 SE)
• either-2.5, n=100, chi2(5), vs A2: diff -0.0109 (-14.4 SE)
• either-2.5, n=100, chi2(5), vs W: diff -0.0202 (-20.3 SE)
• minp-cal, n=100, chi2(5), vs A2: diff -0.0065 (-10.6 SE)
• minp-cal, n=100, chi2(5), vs W: diff -0.0158 (-17.9 SE)

Verdict per rule 2 (bonf-maxp and either-2.5)

Neither required combination is omnibus-safe. The two combinations are numerically identical to within one rejection in a few thousand (the max-p rule and the either rule are the same event up to tie-breaking), so each cell below holds for both.

• vs plain AD: win on t(3) at every n, t(5) at n=50 and 100 (tie at n=25), and the contamination mixture at every n; loss on chi2(5) at every n (-19, -27, -14 SE). That matches 91202 and 91404 cell for cell.
• vs Shapiro-Wilk: loss on chi2(5) at every n (-51, -56, -20 SE); also loss on t(5) at n=25, on the contamination mixture at n=25, 50, 100, and win only on t(3) at n=50, 100 and t(5) at n=50, 100. The n=100 contamination cell was a tie at 91404 (-1.5 SE) and is a loss here (-4.6 SE): it sits on the line across seeds.
• vs S2 alone: below on every heavy-tail and contamination cell (by 14 to 44 SE), above S2 on chi2(5) at n=50 and 100, below at n=25. The combination does not stay within two SE of S2 anywhere.

Omnibus-safe verdict: no. Losing cells, both combinations: chi2(5) vs A2 at n=25, 50, 100; chi2(5) vs W at n=25, 50, 100; mixture vs W at n=25, 50, 100; t(5) vs W at n=25.

What the extra minp-cal row says

Calibrating the min-p statistic to exact 5% size recovers most of what Bonferroni throws away: minp-cal beats both AD and Shapiro-Wilk on every t(3), t(5) and contamination cell at every n (smallest margin +4.3 SE, mixture vs W at n=100), and on chi2(5) it beats AD at n=25 (+5.5 SE) while still losing to AD at n=50 and 100 (-11 SE each) and to Shapiro-Wilk at every n (-36, -48, -18 SE). It stays 13-32 SE below S2 alone on the heavy-tail laws. So about half of the chi2(5) gap to AD at n=25 to 100 is the Bonferroni size penalty, and the remaining half, plus the whole gap to Shapiro-Wilk, is structural: A2 is not skew-sensitive enough to compensate for S2 on a right-skewed law. A partner statistic that is weakly correlated with S2 under the null and sensitive to skew (sample skewness, or Shapiro-Wilk itself) is the next thing to try, with the same size-calibrated min-p rule.