leetcrypt
f920516c69
Operator-authorized (D1: freeze + run). New harness analysis/rq2p3_confirm.py
runs the frozen §4 battery entirely offline: all 9 sweep cells (B∈{2,4,8} ×
alpha∈{0,1,2}; B=50 anchor excluded) × R=30 × C=50 = 13,500 bridged circuits.
RQ2 entropy is analytic and pool bridge assignment is deterministic from the
seed, so no engine, no traffic, no grid, $0 — the same offline reconstruction
path confirm_load_rq2 uses for the lead RQ2. Seeds per frozen §6: run =
derive_seed(S0=20260719 ‖ cell_id ‖ run_index), per-circuit = the live
executor's real _circuit_seed rule (byte-identical). The frozen instruments
(stats.py, confirm.py, confirm_load_rq2.py, assembler.py) are UNCHANGED — the
only new code is seed enumeration, run-as-unit grouping, and an OLS-slope helper.
Frozen §8 analysis (effect + BCa 95% CI, never a bare p; 10,000 resamples;
α=0.05; run-level cluster bootstrap):
H1 pooled Spearman ρ(c_i,H_i) = +0.6244, CI [+0.5941, +0.6545] -> mix
H2 OLS dose-response slope = +0.7052, CI [+0.6195, +0.7903] -> mix (n=270)
H3 joint -> RESOLVED = mix (both exclude 0, agree in sign)
Holm over own family {H1-pooled, H2-slope}, size 2: both reject.
Honest disclosure (as commanded): the §7 dry pass already previewed this mix
(ρ 0→+0.838); the battery quantifies an effect already visible at calibration;
the two-sided pre-commitment stood. The +ρ MIX qualifies/corrects the lead
RQ2-P1 "shrink" headline as a unique-bridge artifact — a shared finite bridge
pool RAISES the per-circuit anonymity set, the opposite of the naive funnel.
That correction is the finding, reported openly. The lead RQ2-P1 result is NOT
re-litigated; any pool ΔH is exploratory (prereg §1).
Sealed immutably under output/ (gitignored → anchors force-added):
rq2p3-confirmatory-results.json (SHA-256 5fdcb379d8a2…) + SHA256SUMS;
byte-identical on re-run. Both prereg SHAs verified intact (lead f22331a72e…,
RQ2-P3 8db4e8a7ac60…). Tests: tests/test_sor_rq2p3_confirm.py 7 passed; full
SOR suite 201 passed, no regression. Defensive-measurement instrument only;
worktree-only on feat/sor-consent-relay.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
97 lines
4.7 KiB
Python
97 lines
4.7 KiB
Python
"""RQ2-P3' confirmatory-harness pins — SYNTHETIC / OFFLINE only.
|
||
|
||
These exercise the OFFLINE confirmatory runner's structure, seed provenance, and
|
||
determinism — NOT the substantive finding (which is the sealed record). Pins:
|
||
(a) the confirmatory cell set is exactly the 9 frozen §4 sweep cells (B∈{2,4,8}×
|
||
alpha∈{0,1,2}), the B=50 calibration anchor excluded, and stays DISJOINT from the
|
||
frozen lead lattice; (b) per-circuit seeds are the confirmatory executor's real rule
|
||
``executor._circuit_seed`` (so the offline circuits are byte-identical to what a live
|
||
executor would persist); (c) the OLS-slope helper is a real cov/var slope; (d) the
|
||
full analysis is byte-deterministic (same S0 → identical results) and well-formed —
|
||
effect + BCa CI for H1 and H2 (never a bare p), H3 joint sign/exclusion logic, and
|
||
Holm over the study's OWN family {H1-pooled, H2-slope} at family_size = 2.
|
||
"""
|
||
|
||
import math
|
||
|
||
from cmd_chat.sor.analysis import rq2p3_confirm as rc
|
||
from cmd_chat.sor.battery import derive_seed, enumerate_cells
|
||
from cmd_chat.sor.executor import _circuit_seed
|
||
|
||
|
||
# --- cell set --------------------------------------------------------------- #
|
||
def test_confirmatory_cells_are_the_nine_sweep_cells_no_anchor():
|
||
cells = rc.confirmatory_cells()
|
||
assert len(cells) == 9
|
||
bs = sorted(int(c.factors["pool_B"]) for c in cells)
|
||
assert bs == [2, 2, 2, 4, 4, 4, 8, 8, 8] # 3 alphas each
|
||
alphas = sorted({float(c.factors["pool_alpha"]) for c in cells})
|
||
assert alphas == [0.0, 1.0, 2.0]
|
||
assert all(c.factors["topology"] == "bridge-federated-pool" for c in cells)
|
||
assert all(int(c.factors["pool_B"]) != 50 for c in cells) # B=50 anchor excluded
|
||
|
||
|
||
def test_confirmatory_cells_disjoint_from_frozen_lead_lattice():
|
||
lead = {c.cell_id for c in enumerate_cells()}
|
||
assert lead.isdisjoint({c.cell_id for c in rc.confirmatory_cells()})
|
||
|
||
|
||
# --- seed provenance matches the live executor rule ------------------------- #
|
||
def test_per_circuit_seeds_use_the_frozen_executor_rule():
|
||
cell = rc.confirmatory_cells()[0]
|
||
run_seed, seeds, specs = rc._assemble_run(cell, 0, c=8)
|
||
assert run_seed == derive_seed(cell.cell_id, 0)
|
||
assert seeds == [_circuit_seed(run_seed, i) for i in range(8)] # byte-identical rule
|
||
assert len(specs) == 8 and all(s.bridge_present for s in specs)
|
||
|
||
|
||
# --- OLS slope helper ------------------------------------------------------- #
|
||
def test_ols_slope_is_cov_over_var():
|
||
# y = 3x + 1 → slope 3 exactly; a single point / flat x → 0 (never fabricated).
|
||
pts = [(x, 3.0 * x + 1.0) for x in (0.0, 1.0, 2.0, 5.0)]
|
||
assert abs(rc._ols_slope(pts) - 3.0) < 1e-9
|
||
assert rc._ols_slope([(1.0, 9.0)]) == 0.0
|
||
assert rc._ols_slope([(2.0, 1.0), (2.0, 9.0)]) == 0.0 # zero-variance x
|
||
|
||
|
||
# --- full analysis: deterministic + well-formed ----------------------------- #
|
||
def test_run_confirmatory_deterministic_and_wellformed():
|
||
a = rc.run_confirmatory(r=3, c=10, n_resamples=200)
|
||
b = rc.run_confirmatory(r=3, c=10, n_resamples=200)
|
||
assert a == b # same S0 → byte-identical
|
||
|
||
assert a["offline_deterministic"] and a["no_engine_no_traffic_no_grid"]
|
||
assert a["n_cells"] == 9 and a["base_seed_S0"] == 20260719
|
||
assert a["n_pooled_bridged_circuits"] == 9 * 3 * 10
|
||
|
||
h1 = a["results"]["H1_pooled_spearman"]
|
||
h2 = a["results"]["H2_dose_response_slope"]
|
||
# effect + CI present for both (never a bare p); p is only carried for Holm order.
|
||
for h in (h1, h2):
|
||
assert {"point", "ci_lo", "ci_hi", "p_for_holm"} <= set(h)
|
||
assert h["ci_lo"] <= h["point"] <= h["ci_hi"]
|
||
assert h["decision"] in ("funnel", "mix", "inconclusive")
|
||
assert h2["n_points"] == 9 * 3 # per-run dose-response points
|
||
|
||
h3 = a["results"]["H3_joint"]
|
||
# H3 resolved iff both CIs exclude 0 AND signs agree — recompute from H1/H2.
|
||
excl = (h1["ci_lo"] > 0 or h1["ci_hi"] < 0) and (h2["ci_lo"] > 0 or h2["ci_hi"] < 0)
|
||
same = (h1["point"] > 0) == (h2["point"] > 0)
|
||
assert h3["resolved"] == bool(excl and same)
|
||
|
||
|
||
def test_holm_is_over_own_family_of_two():
|
||
a = rc.run_confirmatory(r=3, c=10, n_resamples=200)
|
||
holm = a["results"]["holm_own_family"]
|
||
assert {h["name"] for h in holm} == {"H1-pooled", "H2-slope"}
|
||
# family_size = 2 → the smallest-p test gets multiplier 2, the next gets 1.
|
||
assert sorted(h["multiplier"] for h in holm) == [1, 2]
|
||
|
||
|
||
def test_disclosure_and_freeze_provenance_present():
|
||
a = rc.run_confirmatory(r=2, c=6, n_resamples=50)
|
||
assert a["prereg_frozen"] == "2026-07-21"
|
||
assert a["prereg_sha256_sidecar"] == "docs/rq2p3-mechanism-prereg.sha256"
|
||
assert "two-sided" in a["honest_disclosure"]
|
||
assert "unique-bridge artifact" in a["honest_disclosure"]
|