Cycle 85: concave term transforms on the cycle 75 gate (log(1+ct)/c, sqrt)

Third item of the cycle. The two earlier items (tail-trim-rankweight-sweep.md, tail-trimfrac-invrank-sweep.md) changed which AD terms are kept and stalled at var/mean^2 0.79-0.95: once the top m terms by |draw| rank are dropped, the next term inherits the dominance, and r^-p rank weights move the wrong way. This item leaves every term in and changes how each term is computed, passing t_i = (F_n(i) - Phi(z_i))^2 / Phi(z_i) through a concave map h before summation: n^-1/2 * sum_i w_i h(t_i).

• log(1+ct)/c for c = 1, 10, 30, 50, 100 (the /c keeps h(t) ~ t for the bulk, where ct is small, so the normal-data calibration barely moves; only the few huge terms are compressed).
• sqrt(t).
• log(1+t) combined with the m=ceil(0.1n) trim.
• The rule 1 variants (trimmed sum m=ceil(0.1n), rank-weighted r^-1, r^-2) and the k=0 uncapped baseline, in the same table, on the same draws.

Seed 85002, 4000 sims per n per law (normal and t(3)), n = 25, 50, 100. Script: stuff/tail-logterm-transform-sweep.py (pure python, about 4 s). Raw rows: stuff/tail-logterm-transform-sweep.csv. Seed and stream match the earlier two items, so the k0/trim/rw1/rw2 rows reproduce their numbers exactly. Power is P(stat > normal-data p95 of the same variant at the same n) on t(3) data; loss is against k=0 at each n. Within a variant the power and loss columns repeat across the n=25 and n=50 rows. "Top-1% AD share of top 3 draws" is the share of the transformed, weighted statistic carried by the 3 largest |draws|, averaged over the top 1% of t(3) sims by statistic value.

Answer to the direction

Yes, one variant meets both halves: log(1+30t)/30. var/mean^2 is 0.497 at n=25 and 0.572 at n=50 (bound 2/3), and normal-calibrated t(3) power is within 0.02 of the uncapped gate at every n (-0.012 at n=25, +0.019 at n=50, -0.002 at n=100). Its top-3-draw share in the top 1% of sims falls from 0.99-1.00 (k=0) to 0.27-0.31, so the ordered tail still carries most of the statistic, but no longer all of it.

Why this works where trimming did not: the tail excess is a single term of order F^-1 from the most negative standardized draw, and whichever term is largest after a trim inherits it. A concave map caps the contribution of any one term at roughly log(c t_max)/c without zeroing the small terms that carry the t(3) signal, so power is untouched while the variance collapses. The scale c trades the two halves against each other:

The power loss at n=25 grows monotonically with c while var/mean^2 falls; c in roughly 20-45 is the window where both hold on this seed. sqrt(t) is too weak a compression (var/mean^2 78 at n=50, one sim with a F~1e-12 term still dominates). Trimming on top of the log transform brings back the trim's 0.03-0.07 power loss and gains nothing on var/mean^2, so the two fixes do not stack.

Caveat: the +0.019 power at n=50 for c=30 and the -0.021 at n=25 for c=50 are each within one Monte Carlo standard error (about 0.008 at 4000 sims) of the 0.02 line, so the exact edge of the c window is seed-dependent; the qualitative ordering (c=10 fails on variance, c=100 fails on power, c~30 passes) is not. A re-run at 20000 sims with c in 20, 25, 30, 35, 40 would pin the window.

Rule 1 table: t(3), k=0 is the uncapped baseline

Normal data, same variants (calibration rows; power is 0.05 by construction)

Rule 2 verdict and frontier

• trimmed sum (drop top m=ceil(0.1n)): var/mean^2 n=25 0.8634, n=50 0.9532 (bound missed); power loss n=25 -0.067, n=50 -0.030, n=100 -0.019 (within 0.02: no); meets both: no
• rank-weighted r^-1: var/mean^2 n=25 53.42, n=50 536.9 (bound missed); power loss n=25 -0.046, n=50 -0.120, n=100 -0.196 (within 0.02: no); meets both: no
• rank-weighted r^-2: var/mean^2 n=25 55.04, n=50 536.9 (bound missed); power loss n=25 -0.084, n=50 -0.185, n=100 -0.307 (within 0.02: no); meets both: no
• log(1+t) terms: var/mean^2 n=25 2.522, n=50 3.246 (bound missed); power loss n=25 -0.004, n=50 +0.002, n=100 +0.000 (within 0.02: yes); meets both: no
• log(1+10t)/10 terms: var/mean^2 n=25 0.7711, n=50 0.9051 (bound missed); power loss n=25 -0.007, n=50 +0.014, n=100 -0.003 (within 0.02: yes); meets both: no
• log(1+30t)/30 terms: var/mean^2 n=25 0.4974, n=50 0.5719 (bound met); power loss n=25 -0.012, n=50 +0.019, n=100 -0.002 (within 0.02: yes); meets both: YES
• log(1+50t)/50 terms: var/mean^2 n=25 0.4065, n=50 0.4669 (bound met); power loss n=25 -0.021, n=50 +0.015, n=100 -0.002 (within 0.02: no); meets both: no
• log(1+100t)/100 terms: var/mean^2 n=25 0.308, n=50 0.3546 (bound met); power loss n=25 -0.027, n=50 +0.020, n=100 -0.001 (within 0.02: no); meets both: no
• sqrt(t) terms: var/mean^2 n=25 0.6332, n=50 78.49 (bound missed); power loss n=25 -0.015, n=50 +0.031, n=100 +0.013 (within 0.02: no); meets both: no
• log(1+t) terms, drop top m=ceil(0.1n): var/mean^2 n=25 0.8163, n=50 0.9033 (bound missed); power loss n=25 -0.066, n=50 -0.029, n=100 -0.019 (within 0.02: no); meets both: no
• smallest power loss among variants reaching var/mean^2 <= 2/3 at both n: log(1+30t)/30 terms (worst loss 0.012)

Tail-trim and rank-weight sweep: the trimmed sum gets var/mean^2 under 1 but misses 2/3, and r^-p weighting makes the top draw worse, not better

Answer to the cycle 85 direction (rule 2 verdict). No variant meets both halves. Per variant:

• Trimmed sum (drop top m=ceil(0.1n) AD terms by |draw| rank): no. var/mean^2 0.86 at n=25 and 0.95 at n=50 (bound 2/3 missed, but the only variant to bring it under 1 while keeping the shape of the statistic). Power loss -0.067 at n=25, -0.030 at n=50, -0.019 at n=100 (within 0.02 only at n=100). The top-3 share is 0 by construction (m is 3 and 5, so the top 3 draws are always dropped).
• Rank-weighted r^-1: no. var/mean^2 53 at n=25 and 537 at n=50, no better than uncapped (43, 537). Power loss -0.046, -0.120, -0.196. Top-3 share 0.999 / 1.000.
• Rank-weighted r^-2: no. var/mean^2 55 and 537. Power loss -0.084, -0.185, -0.307. Top-3 share 1.000 / 1.000.
• Smallest power loss among variants reaching var/mean^2 <= 2/3: none reaches it. Among all three, the trimmed sum has the smallest loss at every n and the only var/mean^2 below 1.

Why r^-p fails: the weight on the |draw| of rank r is r^-p, so the single largest draw keeps weight 1 while every other term is shrunk. The variant removes the stabilising mass of the bulk and leaves the dominant extreme term untouched, so the top-3 share rises to 1.000 and at n=50 the statistic is numerically identical to the uncapped gate (one term carries the whole sum). Under normal data the same weighting inflates var/mean^2 from 0.77 to 9.4 and 27 at n=25, so the calibrated critical value collapses and power drops. Weighting the ordered tail the way the direction asked moves in the wrong direction; the trimmed sum is the only tail-wide statistic here that acts on the excess.

Frontier (var/mean^2 at n=25, n=50 ; power loss at n=25, 50, 100):

Compared with cycle 84: trimming m=3 at n=25 costs 0.067 against the winsorize k=3 cost of 0.327, and m=5 at n=50 costs 0.030 against 0.520 for k=5. Dropping the extreme AD terms is far cheaper than flattening the draws because the remaining terms still see the heavy-tailed standardization (the dropped draws still inflate the sample sd, which pulls the bulk z-scores inward and keeps the AD signal). It just does not drop far enough: 0.86 to 0.95, not 2/3.

Inputs (rule 1)

• Statistic: cycle 71/75 lower-tail-weighted Anderson-Darling gate, n^-1/2 * sum_i w_i (F_n(i) - Phi(z_i))^2 / Phi(z_i) on standardized data, same pure-python code as cycles 82 to 84; w_i depends on the |draw| rank r_i (r=1 largest): k0 w=1; trim w=0 for r<=m=ceil(0.1n) else 1; rw1 w=r^-1; rw2 w=r^-2. Standardization is on the full sample in all variants.
• Script stuff/tail-trim-rankweight-sweep.py, output stuff/tail-trim-rankweight-sweep.csv, runs in about 2 s.
• Seed 85002 (per-stream 8500200 + n + {0 t(3), 1 normal}); 4000 sims per n per law; n = 25, 50, 100; t(3) drawn as gauss / sqrt(gamma(1.5, 2/3)) as in cycle 71.
• Share: weighted AD terms of the 3 largest |draw| divided by the weighted AD sum, averaged over the top 1% of t(3) sims by statistic value.
• Power: critical value is the normal-data p95 of the same variant at the same n; power is the fraction of t(3) samples above it. Uncapped baseline power 0.379, 0.567, 0.827 (cycle 84: 0.378, 0.583, 0.835; cycle 83: 0.372, 0.585, 0.834).

Rule 1 table: t(3), k=0 is the uncapped baseline

The power columns are per-n properties of a variant and so repeat across the n=25 and n=50 rows. At n=50 the r^-1 and r^-2 rows match the uncapped row to 4 digits because one sim with a draw of several hundred standard deviations carries the entire mean and variance in all three.

Normal data, same variants (calibration rows; power is 0.05 by construction)

What this suggests next

The trimmed sum is the first variant since cycle 82 to land under var/mean^2 1 at a power cost below 0.07. Two obvious knobs: trim fraction between 0.1 and 0.2 (m=4..5 at n=25, 6..10 at n=50) to see whether 2/3 is reachable before the power loss passes 0.02 at n=100, and an inverted rank weight (w = 1 - r^-p, or w = r/n) that shrinks the extreme terms instead of the bulk, which is what the r^-p variants here should have been if the goal is to damp the top draw.

Cycle 85: trim-fraction sweep and inverted rank weights on the cycle 75 gate

Follow-up to tail-trim-rankweight-sweep.md, which found the m=ceil(0.1n) trimmed sum the cheapest variant so far but short of var/mean^2 at most 2/3, and found r^-p rank weighting moves the wrong way because rank 1 keeps weight 1. This item runs the two follow-ups suggested there:

• Trim fraction sweep: drop the top m=ceil(f n) AD terms by |draw| rank for f = 0.10, 0.15 and 0.20.
• Inverted rank weights that damp the extremes instead of the bulk: w = 1 - r^-1, w = 1 - r^-2, and a linear ramp w = min(1, (r-1)/ceil(0.2n)).

The rule 1 variants (trim f=0.10, r^-1, r^-2) and the k=0 baseline appear in the same table. Each variant is computed on the same draws.

Seed 85002, 4000 sims per n per law (normal and t(3)), at n = 25, 50 and 100. Script: stuff/tail-trimfrac-invrank-sweep.py (pure python, about 3 s). Raw rows: stuff/tail-trimfrac-invrank-sweep.csv. The seed and stream match the earlier item, so the k0/trim/rw1/rw2 rows reproduce its numbers exactly. Power is P(stat > normal-data p95 of the same variant at the same n) on t(3) data. Loss is measured against k=0 at each n. Within a variant, the power and loss columns are the same in the n=25 and n=50 rows.

Rule 1 table: t(3), k=0 is the uncapped baseline

Normal data, same variants (calibration rows; power is 0.05 by construction)

Rule 2 verdict and frontier

• trimmed sum (drop top m=ceil(0.1n)): var/mean^2 n=25 0.8634, n=50 0.9532 (bound missed); power loss n=25 -0.067, n=50 -0.030, n=100 -0.019 (within 0.02: no); meets both: no
• rank-weighted r^-1: var/mean^2 n=25 53.42, n=50 536.9 (bound missed); power loss n=25 -0.046, n=50 -0.120, n=100 -0.196 (within 0.02: no); meets both: no
• rank-weighted r^-2: var/mean^2 n=25 55.04, n=50 536.9 (bound missed); power loss n=25 -0.084, n=50 -0.185, n=100 -0.307 (within 0.02: no); meets both: no
• trimmed sum m=ceil(0.15n): var/mean^2 n=25 0.8262, n=50 0.8794 (bound missed); power loss n=25 -0.065, n=50 -0.021, n=100 -0.020 (within 0.02: no); meets both: no
• trimmed sum m=ceil(0.2n): var/mean^2 n=25 0.7923, n=50 0.8343 (bound missed); power loss n=25 -0.069, n=50 -0.023, n=100 -0.028 (within 0.02: no); meets both: no
• inverted 1-r^-1: var/mean^2 n=25 0.8931, n=50 1.387 (bound missed); power loss n=25 -0.064, n=50 -0.034, n=100 -0.023 (within 0.02: no); meets both: no
• inverted 1-r^-2: var/mean^2 n=25 0.9523, n=50 1.748 (bound missed); power loss n=25 -0.062, n=50 -0.029, n=100 -0.022 (within 0.02: no); meets both: no
• ramp min(1,(r-1)/ceil(0.2n)): var/mean^2 n=25 0.8507, n=50 0.9516 (bound missed); power loss n=25 -0.064, n=50 -0.022, n=100 -0.022 (within 0.02: no); meets both: no
• smallest power loss among variants reaching var/mean^2 <= 2/3 at both n: none

Reading

• Trimming more helps var/mean^2 only slowly, and never reaches 2/3. Going from f=0.10 to 0.15 to 0.20 moves var/mean^2 from 0.863 to 0.826 to 0.792 at n=25, and from 0.953 to 0.879 to 0.834 at n=50. That is about 0.035 to 0.06 per 0.05 of trim fraction. A linear extrapolation reaches 2/3 at n=50 only near f of about 0.35.
• Power has a floor at n=25 that trimming cannot remove. Every damping variant (all trims, both inverted weights and the ramp) loses 0.062 to 0.069 at n=25. The loss does not depend on how much is removed, so the gate loses that power as soon as the top draws are touched at all. At n=50 and 100 the losses fall to 0.02-0.03; f=0.15 comes closest at -0.021 and -0.020, just outside 0.02.
• The inverted weights fix the direction but not the size. 1 - r^-1 and 1 - r^-2 take the top-1% top-3 share down from 0.99-1.00 (k0, r^-p) to 0.12-0.15 at n=25. At n=50, though, the share is 0.40-0.52 and var/mean^2 is 1.39-1.75. The partial weight on ranks 2-3 is still enough to dominate when n is larger. The ramp, which zeros rank 1 and phases ranks 2..ceil(0.2n) in linearly, behaves like a soft f=0.10 trim (0.851 / 0.952) at f=0.15 power cost.
• Frontier (var/mean^2 n=25 / n=50 ; worst power loss over n=25, 50, 100): trim f=0.20 0.792/0.834 ; 0.069. trim f=0.15 0.826/0.879 ; 0.065. ramp 0.851/0.952 ; 0.064. trim f=0.10 0.863/0.953 ; 0.067. 1-r^-1 0.893/1.387 ; 0.064. 1-r^-2 0.952/1.748 ; 0.062. r^-1 53.4/537 ; 0.196. r^-2 55.0/537 ; 0.307.
• Rule 2 answer: no variant meets both halves. No variant reaches var/mean^2 at most 2/3 at both n, so the variant with the smallest power loss among those reaching the bound is none. On the frontier, trim f=0.20 has the lowest var/mean^2 at both n. Trim f=0.15 has the best power at n=50 and 100 among trims.
• Implication: the power cost at n=25 comes from removing any of the tail, and var/mean^2 falls only slowly with the trim fraction. Under t(3), any whole-tail reweighting of this AD gate trades about 0.065 of n=25 power for a var/mean^2 that stays above 0.79. Reaching 2/3 likely needs a change in how terms are computed, for example a bounded per-term transform such as log(1+term), rather than in which terms are kept.