Files

T

leetcrypt 6d61c53152 RQ1+RQ2 start-line: live per-cell condition assembler + guarded launcher + Holm ratification

Wire the live per-cell condition assembler (cmd_chat/sor/assembler.py): compose
the existing R1/R4/R5/R6 pieces into a condition-encoding CircuitSpec per frozen
§2 cell, so a cell_id maps to a genuinely distinct, ForwarderPlan-gated
(engine != local) isolated circuit rather than the plain 3-hop control:
  - RQ1 bridge-on / on+padding insert a live bridge hop (+ R1 PADDING stream);
  - RQ2 bridge-/directory-federated genuinely span >= 2 houses
    (federation.select_federated_path, split-knowledge).
Plans only: opens no socket, moves no traffic, stands up no engine.

battery.assembler_dry_check validates on FIXTURES (6/6 cells distinct
fingerprints, all isolation-gated, same (cell,seed) reproduces, RQ1 bridge +
padding arms live, RQ2 federation >= 2 houses); write-once, kept out of any
confirmatory data dir. confirmatory_run preflight now runs it; the --operator-go
path no longer refuses cells as "not wired" — triple-lock + green preflight HOLD
on grid completion, then surface the operator's immutable data-run gate. No
confirmatory cell is ever fabricated.

Start-line instrument-validation gate (cmd_chat/sor/gate.py), grid + containment
pin (grid.py), and the guarded launcher (confirmatory_run.py) land alongside.

Holm: hold family_size = 7, report the 4 RQ1/RQ2 tests -> multipliers 7,6,5,4.
This is the pre-registration, not a deviation (prereg §6 [APPROVAL] size-7 family
+ D6 disclosure in the lead paper); docs/stage-05-holm-clarification.md marked
RATIFIED. RQ3-fold and alpha-split declined.

Containment intact; prereg untouched (SHA f22331a72e...); worktree-only. output/
gitignored (runtime artifacts, never source). Full SOR suite 156 passed. No
confirmatory data collected — the live data run remains the human gate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-07-19 20:49:12 -07:00

4.8 KiB

Raw Blame History

Stage-05 analysis clarification — Holm–Bonferroni over the frozen family of 7

Status: RATIFIED by operator (Andre), 2026-07-19. Worktree clarification note only — NOT a deviation. This restates, for the harness, a decision the frozen prereg already fixes; it does not edit the frozen prereg (sor-consent-prereg.md, SHA f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b).

Authority (this IS the freeze, not a new choice)

The size-7 family and its disclosure in the lead paper are pre-registered text, so holding m = 7 for the reported RQ1/RQ2 subset is compliance, not deviation:

Prereg §6 — Multiple-comparison correction [APPROVAL] (prereg lines ~218–222): "the confirmatory hypothesis tests across all three RQ families are corrected together by Holm–Bonferroni … Test family (7 confirmatory tests): {RQ1-P1, RQ1-P2, RQ2-P1, RQ2-P3, RQ3-P1-perf, RQ3-P1-latency, RQ3-P2}." The family is frozen at 7.
Prereg D6 (prereg line ~317): "the shared prereg + Holm-corrected family are disclosed in whichever papers result (which is more rigorous)." The lead paper discloses the same size-7 family even while reporting only the RQ1/RQ2 subset.

Correcting the reported 4 against m = 4 would shrink the frozen family after the fact — p-hacking the multiplicity down, contrary to §6. Holding m = 7 is the only reading consistent with the frozen text.

The question

The prereg §6 pre-registers a confirmatory family of 7 hypothesis tests:

RQ1-P1, RQ1-P2, RQ2-P1, RQ2-P3, RQ3-P1-perf, RQ3-P1-latency, RQ3-P2

The lead paper (G4 + RQ1 + RQ2, prereg D6) reports only the first 4 (RQ1/RQ2); RQ3's three tests are the severable follow-on (§8). So: when we Holm-correct the 4 reported p-values, do we correct against m = 7 (the frozen family) or m = 4 (the reported subset)?

Correcting against m = 4 would be p-hacking the multiplicity down: the family was frozen at 7 before data, and shrinking it after the fact — because RQ3 is being reported later — inflates the reported significance of the RQ1/RQ2 subset. That breaks the pre-registration.

Decision (implemented default)

Hold the family size at m = 7 for every reported test. The reported subset never shrinks the correction to itself.

Concretely, Holm step-down (stats.holm_bonferroni, confirm.apply_holm):

All reported p-values are sorted ascending and assigned ranks k = 1..R (R = 4 reported here).
Each gets multiplier m − k + 1 with m = FROZEN_FAMILY_SIZE = 7.
So the 4 reported tests receive multipliers 7, 6, 5, 4 (smallest p → ×7). The step-down enforces monotone non-decreasing adjusted p-values.
holm_bonferroni raises if family_size < #reported — you can never accidentally correct against fewer than the tests you are reporting.

This is a conservative embedding: the 4 reported tests are treated as occupying the 4 smallest-multiplier slots (7,6,5,4) of the size-7 family rather than the true unknown slots they would occupy if all 7 p-values were in hand. The adjusted p-values are therefore ≥ the values a full simultaneous size-7 Holm run would assign to these same tests. We over-correct, never under-correct — the honest direction for a subset report.

Covered by tests/test_sor_confirm.py::test_apply_holm_corrects_against_family_of_seven (top multiplier == 7; multipliers == [4,5,6,7]) and tests/test_sor_stats.py (family-size monotonicity + ValueError when family_size < reported).

Gate vs. Holm ordering (not conflated)

Every reported decision in §6 is a CI gate (RQ1-P1 CI excludes 0.5; RQ1-P2 ΔAUC CI > 0; RQ2-P1 ΔH two-sided CI sign; RQ2-P3 Spearman ρ CI). The bootstrap p-values feed the Holm step-down only to order it — no reported claim rests on a bare p. Holm is the family-wise multiplicity guard layered on top of the pre-registered CI gates, exactly as §6 / rigor-standards §Statistics require.

Operator ruling (2026-07-19)

The operator (Andre) ratified the conservative embedding: hold family_size = 7, report the 4 RQ1/RQ2 tests, multipliers 7, 6, 5, 4. This is exactly the frozen §6 / D6 text above — the harness default now matches the pre-registration.

Two alternatives were considered and explicitly declined:

Fold RQ3's three cells back in and report all 7 simultaneously. Statistically clean, but RQ3 was dropped for the lead paper (prereg D6) and would need the paid/agent selector-arm decisions. Declined: do not fold RQ3 back.
Full α-splitting / hierarchical gatekeeping. More powerful, but requires a pre-specified α-allocation the frozen prereg does not contain. Declined: not available without a prereg change (which the freeze forbids).

No open human action remains: the implemented default is the ratified one.

4.8 KiB Raw Blame History Unescape Escape