hack-house/docs/stage-05-rq1p2-pairing-clarification.md

Stage-05 analysis clarification — RQ1-P2 paired-bootstrap pairing = by run_index

Status: RATIFIED-by-derivation, 2026-07-20 — PROPOSED for operator (Andre) confirmation (low-risk, freeze-faithful, non-blocking). Worktree clarification note only — NOT a deviation. This reads an implied unit out of the frozen prereg; it does not edit the frozen prereg (sor-consent-prereg.md, SHA f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b).

Made while BLIND (battery mid-collection, 0 RQ1 confirmatory statistics computed, inspected, or reported). Like the Holm clarification — and unlike the RQ2-posterior clarification — this adds no construct: it restates the pairing the frozen text already implies for the §6 paired bootstrap.

Distinction from the RQ2 case. The RQ2 note filled a genuine gap (the prereg pinned a per-circuit posterior but no construction rule → it needed operator ratification). This note is the Holm case: the frozen text fixes the answer; naming it is compliance, not a new methodological choice. Hence "RATIFIED-by-derivation" — operator confirmation is a courtesy check, not a gate.

The question

Frozen §6 RQ1-P2 mandates a paired bootstrap of a padding-efficacy difference:

• §6 (Analysis), RQ1-P2 (prereg L197–198): "ΔAUC = AUC(bridge-on, no-pad) − AUC(bridge-on, +pad); paired bootstrap 95% CI. Padding effective iff ΔAUC CI > 0."

"Paired" is mandated — an unpaired / two-sample bootstrap is out. But a single physical circuit is run under either the no-pad or the +pad arm, never both (they are different §2 cells with different cell_ids → different per-run seeds). So: what is the paired unit? Earlier this was held as a "choice" on the grounds that per-arm seeds differ so nothing pairs — but that conflates seed-pairing with run-pairing.

Authority (the freeze fixes the unit)

Three frozen facts jointly determine the pairing:

• §4 unit of analysis (prereg L108): "RQ1 — the (entry, exit) circuit-segment pair (AUC over the pair set)." The RQ1 statistic is an arm/cell-level set AUC, not a per-circuit quantity. So the pairable unit is never the individual circuit — it is the thing that yields one AUC.
• §4 repetition (prereg L111): "R = 30 independent seeded runs per design cell." Both the no-pad and the +pad arms have the same balanced R = 30 runs, indexed i ∈ {0..29}. This is the only structure that is 1-to-1 across the two arms.
• §3 confounds L97 / executor design (interleaving): run order is randomized and controls interleaved to bracket per-session thermal / background-load drift on the grid. Run index i therefore indexes a shared replicate / session position whose nuisance the pairing is meant to remove.

Pairing by run index is the only balanced pairing the R = 30 interleaved design supports, and it is exactly the nuisance-removal a paired bootstrap exists for.

Decision (freeze-derived, implemented)

RQ1-P2 pairs by run index i ∈ {0..29}. For each run index i:

1. AUC_nopad(i) = the correlator AUC over run i's (entry, exit) pair set in the bridge=on (no-pad) cell;
2. AUC_pad(i) = the AUC over run i's pair set in the bridge=on+padding cell;
3. ΔAUC_i = AUC_nopad(i) − AUC_pad(i) — one paired difference per run index.

The paired bootstrap 95% CI resamples the run-index paired unit (each unit carries both arms' pair sets for that i); the reported effect size is the §6 arm-level ΔAUC = AUC(no-pad) − AUC(+pad) and the gate is unchanged: padding effective iff the CI > 0. Realized as one confirm.PairedCircuit per run index fed to the frozen confirm.rq1_p2_padding decision function (the §6 decision layer is not modified) — the loader only shapes the paired units.

Validity note. Pairing removes the shared replicate/session nuisance under interleaving; it never invalidates the CI. If replicate correlation happens to be 0, the paired bootstrap reduces to ~unpaired conservativeness — so pairing is weakly dominant and safe. Differing per-arm seeds are irrelevant: the pairing is over the run position, not the seed.

Graceful degradation (no R-dropping). All R = 30 run indices are attempted; a run index missing one arm (e.g. an integrity-quarantined run, §6 exclusion rule) is simply not paired — the frozen R is not silently redefined, and if too few paired units remain the test reports inconclusive (never a shrink-to-fit).

Covered by tests/test_sor_confirm_load_rq1p2.py (SYNTHETIC pcaps only): 30 paired ΔAUC_i from per-run AUCs, paired-bootstrap CI via confirm.rq1_p2_padding, and graceful pairing when a run is missing an arm.

Blinding (binding)

Ratification-by-derivation unblocks the CODE, not the results (prereg §2). The collect_rq1_p2_paired real-data entrypoint is BLIND-GATED — it must not run on a confirmatory data dir until the full battery completes (battery-results.json present, or all 180 cell-runs carry metrics.json). The pairing logic is unit-tested on synthetic pcaps only, mirroring the RQ1-P1 loader (8224a38) and the RQ2 loader (bd7e376).

Operator confirmation (requested, non-blocking)

Andre: this is a freeze-derived reading (the Holm pattern), not a new construct. Confirm at leisure; no RQ1-P2 number is computed until the battery completes regardless. If you would rather pair differently (e.g. pool to a single arm-level ΔAUC with an unpaired CI), that would be a deviation (§8) since §6 says paired — flag it and it goes in sor-consent-deviations.md.