SS3(RQ1-P2): freeze-derived run_index paired bootstrap + loader wiring
Pairing ruling (docs/stage-05-rq1p2-pairing-clarification.md, RATIFIED-by- derivation, made while BLIND): the frozen §6 L197-198 "paired" bootstrap pairs the bridge-on (no-pad) and bridge-on+padding arms BY RUN INDEX — the only balanced pairing the R=30 (§4 L111) interleaved (§3 L97) design supports; the §4 L108 RQ1 unit is AUC over the (entry,exit) pair set, so the pairable unit is the run, not the circuit. Prereg SHA untouched (f22331a72e…). confirm_load.collect_rq1_p2_paired feeds one confirm.PairedCircuit per shared run_index into the FROZEN confirm.rq1_p2_padding (§6 arm-level ΔAUC preserved); per_run_delta_aucs exposes the auditable per-run ΔAUC_i; missing-arm indices are unpaired (frozen R not redefined). Unit-tested on SYNTHETIC pcaps only (prereg §2 blinding — unblocks CODE not RESULTS); full SOR suite 176 passed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
# Stage-05 analysis clarification — RQ1-P2 paired-bootstrap pairing = by run_index
|
||||
|
||||
**Status: RATIFIED-by-derivation, 2026-07-20 — PROPOSED for operator (Andre)
|
||||
confirmation (low-risk, freeze-faithful, non-blocking).** Worktree clarification
|
||||
note only — **NOT a deviation**. This *reads an implied unit out of* the frozen
|
||||
prereg; it does **not** edit the frozen prereg (`sor-consent-prereg.md`, SHA
|
||||
`f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b`).
|
||||
|
||||
**Made while BLIND** (battery mid-collection, **0 RQ1 confirmatory statistics
|
||||
computed, inspected, or reported**). Like the Holm clarification — and *unlike* the
|
||||
RQ2-posterior clarification — this adds **no construct**: it restates the pairing
|
||||
the frozen text already implies for the §6 *paired* bootstrap.
|
||||
|
||||
> **Distinction from the RQ2 case.** The RQ2 note filled a genuine gap (the prereg
|
||||
> pinned a per-circuit posterior but no construction rule → it needed operator
|
||||
> *ratification*). This note is the **Holm case**: the frozen text fixes the
|
||||
> answer; naming it is compliance, not a new methodological choice. Hence
|
||||
> "RATIFIED-by-derivation" — operator confirmation is a courtesy check, not a gate.
|
||||
|
||||
## The question
|
||||
|
||||
Frozen §6 RQ1-P2 mandates a **paired** bootstrap of a padding-efficacy difference:
|
||||
|
||||
- **§6 (Analysis), RQ1-P2** (prereg L197–198): *"ΔAUC = AUC(bridge-on, no-pad) −
|
||||
AUC(bridge-on, +pad); **paired** bootstrap 95% CI. Padding effective iff ΔAUC CI
|
||||
**> 0**."*
|
||||
|
||||
"Paired" is **mandated** — an unpaired / two-sample bootstrap is out. But a single
|
||||
physical circuit is run under **either** the no-pad **or** the +pad arm, never both
|
||||
(they are different §2 cells with different `cell_id`s → different per-run seeds).
|
||||
So: **what is the paired unit?** Earlier this was held as a "choice" on the grounds
|
||||
that per-arm *seeds differ so nothing pairs* — but that conflates **seed-pairing**
|
||||
with **run-pairing**.
|
||||
|
||||
## Authority (the freeze fixes the unit)
|
||||
|
||||
Three frozen facts jointly determine the pairing:
|
||||
|
||||
- **§4 unit of analysis** (prereg L108): *"RQ1 — the (entry, exit) circuit-segment
|
||||
pair (**AUC over the pair set**)."* The RQ1 statistic is an **arm/cell-level set
|
||||
AUC**, *not* a per-circuit quantity. So the pairable unit is never the individual
|
||||
circuit — it is the thing that yields one AUC.
|
||||
- **§4 repetition** (prereg L111): *"**R = 30** independent seeded runs per design
|
||||
cell."* Both the no-pad and the +pad arms have the **same balanced R = 30** runs,
|
||||
indexed `i ∈ {0..29}`. This is the only structure that is **1-to-1 across the two
|
||||
arms**.
|
||||
- **§3 confounds L97 / executor design** (interleaving): run order is *randomized
|
||||
and controls interleaved* to bracket **per-session thermal / background-load
|
||||
drift** on the grid. Run index `i` therefore indexes a **shared replicate /
|
||||
session position** whose nuisance the pairing is meant to remove.
|
||||
|
||||
Pairing by run index is the **only balanced pairing the R = 30 interleaved design
|
||||
supports**, and it is exactly the nuisance-removal a *paired* bootstrap exists for.
|
||||
|
||||
## Decision (freeze-derived, implemented)
|
||||
|
||||
**RQ1-P2 pairs by run index `i ∈ {0..29}`.** For each run index `i`:
|
||||
|
||||
1. `AUC_nopad(i)` = the correlator AUC over run `i`'s **(entry, exit) pair set** in
|
||||
the **bridge=on** (no-pad) cell;
|
||||
2. `AUC_pad(i)` = the AUC over run `i`'s pair set in the **bridge=on+padding** cell;
|
||||
3. `ΔAUC_i = AUC_nopad(i) − AUC_pad(i)` — one paired difference per run index.
|
||||
|
||||
The **paired bootstrap 95% CI** resamples the **run-index paired unit** (each unit
|
||||
carries both arms' pair sets for that `i`); the reported effect size is the §6
|
||||
arm-level `ΔAUC = AUC(no-pad) − AUC(+pad)` and the gate is **unchanged: padding
|
||||
effective iff the CI > 0.** Realized as one `confirm.PairedCircuit` per run index
|
||||
fed to the frozen `confirm.rq1_p2_padding` decision function (the §6 decision layer
|
||||
is **not** modified) — the loader only shapes the paired units.
|
||||
|
||||
**Validity note.** Pairing removes the shared replicate/session nuisance under
|
||||
interleaving; it never invalidates the CI. If replicate correlation happens to be
|
||||
0, the paired bootstrap reduces to ~unpaired conservativeness — so pairing is
|
||||
weakly dominant and safe. Differing per-arm *seeds* are irrelevant: the pairing is
|
||||
over the **run position**, not the seed.
|
||||
|
||||
**Graceful degradation (no R-dropping).** All R = 30 run indices are attempted; a
|
||||
run index missing one arm (e.g. an integrity-quarantined run, §6 exclusion rule) is
|
||||
simply not paired — the frozen R is **not** silently redefined, and if too few
|
||||
paired units remain the test reports **inconclusive** (never a shrink-to-fit).
|
||||
|
||||
*Covered by `tests/test_sor_confirm_load_rq1p2.py` (SYNTHETIC pcaps only): 30
|
||||
paired `ΔAUC_i` from per-run AUCs, paired-bootstrap CI via `confirm.rq1_p2_padding`,
|
||||
and graceful pairing when a run is missing an arm.*
|
||||
|
||||
## Blinding (binding)
|
||||
|
||||
Ratification-by-derivation unblocks the **CODE, not the results** (prereg §2). The
|
||||
`collect_rq1_p2_paired` real-data entrypoint is **BLIND-GATED** — it must not run on
|
||||
a confirmatory data dir until the full battery **completes** (`battery-results.json`
|
||||
present, or all 180 cell-runs carry `metrics.json`). The pairing logic is
|
||||
unit-tested on **synthetic** pcaps only, mirroring the RQ1-P1 loader (`8224a38`) and
|
||||
the RQ2 loader (`bd7e376`).
|
||||
|
||||
## Operator confirmation (requested, non-blocking)
|
||||
|
||||
Andre: this is a **freeze-derived** reading (the Holm pattern), not a new construct.
|
||||
Confirm at leisure; no RQ1-P2 number is computed until the battery completes
|
||||
regardless. If you would rather pair differently (e.g. pool to a single arm-level
|
||||
ΔAUC with an unpaired CI), that would be a **deviation** (§8) since §6 says
|
||||
*paired* — flag it and it goes in `sor-consent-deviations.md`.
|
||||
Reference in New Issue
Block a user