Per-n recalibrated gate: does the n-dependence survive?

Answer to the cycle 75 direction. Run by member-2 with stuff/recal-per-n-test.py on cycle 74's stored per-draw rows (null-draws.csv, df3-draws.csv, 2000 draws per seed, both seeds pooled per rule 1). Output: stuff/recal-per-n-test.csv.

Short answer: no, it is not an artefact. With each n's own pooled null p95 as the critical value, the null false-flag rate is flat at 5% by construction, but the df=3 margin and the df=3 flag rate still move across n = 25, 50, 100 under rule 2's step-and-interval test. Only the null rate stops moving, and it stops moving because recalibration pins it.

Per-n critical values (pooled null p95, rule 1)

Recalibrated values with bootstrap 95% intervals (2000 paired resamples, seed 65076)

Rule 2 step-and-interval test on the recalibrated values

Both steps of the df=3 flag rate and the df=3 margin exceed the wider adjacent half-width by an order of magnitude, so the result is not close.

What this says about cycle 74

1. The null false-flag tracking n (12%, 4.3%, 1.4%) was entirely the held critical. It is 5% at every n once each n uses its own p95. That part of cycle 74's finding was miscalibration, as the learning said, and the fix is per-n recalibration, not a different gate.
2. The margin's n-dependence was never an artefact. Cycle 74 already defined margin as median/null p95 *at that n*, so its values (0.70, 1.32, 2.84) are exactly what rule 1 recalibration produces; nothing changes. The contrast row shows that holding the n=50 critical would have *shrunk* the spread (0.94, 1.27, 1.97), not created it. The margin grows with n because the df=3 median stat falls more slowly with n than the null p95 does, which is a property of the statistic, not of the critical value.
3. Power moves with n at fixed 5% false-flag rate: 36%, 60%, 84%. This is the ordinary behaviour of a consistent test and is the third quantity the direction asked about; it moves.
4. The cycle 71 value 2.5 is still not reproduced at n=50 under recalibration (1.316, interval [1.251, 1.396]). It sits between n=50 and n=100.

Caveats

• The null rate's bootstrap interval is degenerate ([0.0495, 0.0500]) because the critical is recomputed from the same resampled null rows; that is the by-construction pin, not evidence of precision. The interesting spread is on the df=3 quantities.
• Both seeds are pooled per rule 1; cycle 74's per-seed split is not repeated here. The pooled n=50 critical (0.058392) sits between the two per-seed values (0.060596 and 0.055337), as expected.
• Reproduce: python3 -I stuff/recal-per-n-test.py > stuff/recal-per-n-test.csv (about 10 s, pure python).

Audit of the per-n recalibrated gate (member-3)

Audit of stuff/recal-per-n-test.py and stuff/recal-per-n-answer.md against rule 1 (per-n critical from the stored rows, both seeds pooled) and rule 2 (step-and-interval test applied to the recalibrated values only). Checked by an independent recompute from null-draws.csv and df3-draws.csv with no shared code: a separate quantile function, statistics.median for the df=3 median, and per-seed splits the script never computes.

Verdict: the script complies with both rules and its numbers reproduce. The answer's conclusion stands: the null rate's n-dependence in cycle 74 was the held critical; the df=3 margin and flag rate move across n on their own.

Rule 1 check: critical read from the stored rows, per n, pooled

• load() keys rows by n only, so both seeds (65068, 65069) pool into one list of 4000 per n. Rule 1 satisfied.
• stats() computes crit = pct(null, 0.95) from that pooled list. The held value HELD is used only for the margin_held contrast row, never for null_rate, df3_rate or margin. Rule 1 satisfied.
• Per-seed n=50 p95s from the rows are 0.060598 and 0.055343. The answer quotes 0.060596 and 0.055337; the 2e-6 and 6e-6 gaps are quantile interpolation differences against cycle 68/74's definition, not a data mismatch. The pooled 0.058392 sits between them as the answer says.

Reproduction of the recalibrated values

All twelve values agree to four decimals. The null rate equals exactly 200/4000 at each n, which is what a linearly interpolated p95 over 4000 rows must give, confirming the "5% by construction" claim.

Rule 2 check: test applied to recalibrated values

• The moved table takes v from res[n][q][0], the point value at that n's own critical, for null_rate, df3_rate and margin. Rule 2 satisfied.
• Sign test and both half-width comparisons are implemented as rule 2 states (wider of the two adjacent intervals, each step must exceed it).
• margin_held also appears in the moved table. Rule 2 says the test applies to the recalibrated values, "never the held n=50 ones". The script labels that row as contrast only and the answer does not count it as a finding, so this is a presentation issue, not a violation. Future runs should drop it from the moved table or mark it excluded.
• Steps dwarf the half-widths: df=3 rate steps 0.237 and 0.240 against half-widths of about 0.02; margin steps 0.615 and 1.527 against 0.073 and 0.180. No borderline call.

Sensitivity: quantile definition

Nearest-rank p95 (the 3800th of 4000 sorted rows) instead of linear interpolation gives criticals 0.081439, 0.058391, 0.041937 and margins 0.7007, 1.3159, 2.8426. The conclusion does not depend on the quantile convention.

Residual caveats

1. The bootstrap null-rate interval is degenerate because the critical is recomputed from the resampled null rows. This is correct for the by-construction claim but means rule 2's test on null_rate is trivially "not moved"; it should not be read as evidence about the gate.
2. Cycle 71's 2.5 at n=50 is still unexplained. The pooled recalibrated margin at n=50 is 1.316 [1.251, 1.396]; the per-seed criticals do not change that enough to reach 2.5 (0.0768/0.0553 = 1.39 at best).
3. Not re-run: the 2000-resample bootstrap itself. Its intervals are taken from recal-per-n-test.csv as given; the point values they bracket all reproduce, and the steps exceed them by an order of magnitude.

Cycle 75 closing note: what the heap-death learning should say

Ties together recal-per-n-answer.md (member-2, the run) and recal-per-n-audit.md (member-3, the check) so the next cycle can cite one file. Numbers are the audited ones; both files agree to four decimals.

One-line verdict

Per-n recalibration removes the null-rate n-dependence and nothing else. The df=3 margin (0.70, 1.32, 2.84) and df=3 power (36%, 60%, 84%) still move across n = 25, 50, 100 under rule 2, by an order of magnitude over the adjacent bootstrap half-widths. Cycle 74's finding splits cleanly:

Why the margin keeps moving

The margin is median(df=3 stat) / null p95 at the same n. Both numerator and denominator fall with n, but the null p95 falls faster, so the ratio grows. That is a property of the statistic under the alternative, the same thing that makes power rise from 36% to 84%. It is not something a critical-value choice can remove, and it is not a defect: a consistent test is supposed to separate better as n grows.

What *was* a defect, and is now closed, is quoting a single critical value across n. The direction's phrase "entirely an artefact" is therefore half right, and the half it is right about is the only half that was ever a calibration problem.

What the audit added

• Independent recomputation of all twelve values (three n, four quantities) matched the run to four decimals.
• Quantile convention (linear interpolation vs nearest-rank) shifts margins by at most 0.0002; no call changes.
• Two presentation notes only: the held-critical contrast row sits inside the "moved" table though rule 2 excludes it (labelled, not counted), and the per-seed split from cycle 74 is not repeated (pooled per rule 1).

Open thread for the next direction

The margin's definition is now settled (median / per-n p95) and the gate is calibrated. What is still untested is whether the n-dependence of the margin is *linear in sqrt(n)* or something else: the three points (0.70, 1.32, 2.84 at n = 25, 50, 100) roughly double per doubling of n, which is faster than sqrt. One more n (200) with the same stored-row protocol would decide it.

Reproduce everything: python3 -I stuff/recal-per-n-test.py (about 10 s).