SS3(RQ1-P2): freeze-derived run_index paired bootstrap + loader wiring
Pairing ruling (docs/stage-05-rq1p2-pairing-clarification.md, RATIFIED-by- derivation, made while BLIND): the frozen §6 L197-198 "paired" bootstrap pairs the bridge-on (no-pad) and bridge-on+padding arms BY RUN INDEX — the only balanced pairing the R=30 (§4 L111) interleaved (§3 L97) design supports; the §4 L108 RQ1 unit is AUC over the (entry,exit) pair set, so the pairable unit is the run, not the circuit. Prereg SHA untouched (f22331a72e…). confirm_load.collect_rq1_p2_paired feeds one confirm.PairedCircuit per shared run_index into the FROZEN confirm.rq1_p2_padding (§6 arm-level ΔAUC preserved); per_run_delta_aucs exposes the auditable per-run ΔAUC_i; missing-arm indices are unpaired (frozen R not redefined). Unit-tested on SYNTHETIC pcaps only (prereg §2 blinding — unblocks CODE not RESULTS); full SOR suite 176 passed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -8,12 +8,15 @@ The candidate true pairing is the diagonal (same circuit); off-diagonal pairs ar
|
||||
unlinked — exactly the §4 "AUC over the (entry, exit) pair set, bootstrap over
|
||||
circuit pairs" unit.
|
||||
|
||||
Deliberately NOT here (held for the operator ruling — see OVERSEER NEEDS-OPERATOR):
|
||||
* **RQ2** — the frozen DV is a *per-circuit adversary sender posterior* (§3/§4),
|
||||
whose construction is not formula-pinned in the prereg and not produced by the
|
||||
instrument; guessing a posterior model post-freeze would be HARKing.
|
||||
* **RQ1-P2** nopad/pad *pairing* convention for the paired bootstrap — the pairing
|
||||
rule across the bridge-on and bridge-on+padding arms is a choice; not guessed.
|
||||
RQ1-P2 (padding efficacy) is wired here too: the frozen §6 **paired** bootstrap
|
||||
pairs the bridge-on (no-pad) and bridge-on+padding arms **by run index** — the only
|
||||
balanced pairing the R=30 interleaved design supports (freeze-derived; see
|
||||
`docs/stage-05-rq1p2-pairing-clarification.md`). Each run index becomes one
|
||||
`confirm.PairedCircuit` fed to the frozen `confirm.rq1_p2_padding`.
|
||||
|
||||
Split out (RQ2 lives in its own loader): **RQ2** — the per-circuit adversary
|
||||
sender posterior (§3/§4) is reconstructed in `confirm_load_rq2.py` per the RATIFIED
|
||||
stage-05 posterior clarification.
|
||||
|
||||
Blinding (prereg §2, binding): callers must **not** run this on a confirmatory
|
||||
data dir until the full battery has completed — no inspection of intermediate
|
||||
@@ -24,7 +27,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Tuple
|
||||
from typing import Dict, List, Sequence, Tuple
|
||||
|
||||
from cmd_chat.sor.analysis.detectors import score_matrix
|
||||
from cmd_chat.sor.executor import DEFAULT_BINS, ExecutorError, bin_pcap_bytes
|
||||
@@ -43,6 +46,7 @@ def classify_run(run_dir: Path) -> Dict[str, object]:
|
||||
"cell_id": d["cell_id"],
|
||||
"rq": d["rq"],
|
||||
"padding_applied": bool(d.get("padding_applied", False)),
|
||||
"run_index": d.get("run_index"), # needed for the §6 RQ1-P2 run-index pairing
|
||||
}
|
||||
|
||||
|
||||
@@ -88,3 +92,63 @@ def collect_rq1_p1_pairs(data_dir: Path, *, bins: int = DEFAULT_BINS, hops: int
|
||||
if info["rq"] == "RQ1" and str(info["cell_id"]).endswith("bridge=on") and not info["padding_applied"]:
|
||||
pairs.extend(reconstruct_run_pairs(run_dir, bins=bins, hops=hops))
|
||||
return pairs
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# RQ1-P2 (padding efficacy) — the frozen §6 PAIRED bootstrap, paired BY RUN INDEX.
|
||||
# Freeze-derived pairing (docs/stage-05-rq1p2-pairing-clarification.md): each run
|
||||
# index i yields one confirm.PairedCircuit carrying that run's no-pad and +pad pair
|
||||
# sets, fed to the frozen confirm.rq1_p2_padding. BLIND-GATED on real data.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _arm_pairs_by_run_index(
|
||||
data_dir: Path, *, want_padding: bool, bins: int, hops: int
|
||||
) -> Dict[int, List[Pair]]:
|
||||
"""Map run_index -> that run's RQ1 (entry,exit) pair set, for the bridge-on arm
|
||||
with (``want_padding``) or without padding. Runs missing a run_index in their
|
||||
persisted metrics.json are skipped (never guessed)."""
|
||||
out: Dict[int, List[Pair]] = {}
|
||||
for run_dir in sorted(p for p in Path(data_dir).iterdir() if p.is_dir()):
|
||||
if not (run_dir / "metrics.json").exists():
|
||||
continue
|
||||
info = classify_run(run_dir)
|
||||
if info["rq"] != "RQ1" or not str(info["cell_id"]).endswith(
|
||||
"bridge=on+padding" if want_padding else "bridge=on"
|
||||
):
|
||||
continue
|
||||
if bool(info["padding_applied"]) != want_padding or info["run_index"] is None:
|
||||
continue
|
||||
out[int(info["run_index"])] = reconstruct_run_pairs(run_dir, bins=bins, hops=hops)
|
||||
return out
|
||||
|
||||
|
||||
def collect_rq1_p2_paired(
|
||||
data_dir: Path, *, bins: int = DEFAULT_BINS, hops: int = 3
|
||||
) -> List["PairedCircuit"]:
|
||||
"""The RQ1-P2 paired units: one :class:`confirm.PairedCircuit` per run index
|
||||
present in **both** the bridge-on (no-pad) and bridge-on+padding arms — the
|
||||
freeze-derived run-index pairing (§6 "paired bootstrap"). Graceful: a run index
|
||||
missing an arm is simply not paired (the frozen R is not silently redefined);
|
||||
the caller reports inconclusive if too few paired units remain. BLIND-GATED —
|
||||
do not call on a confirmatory dir until the battery completes (prereg §2)."""
|
||||
from cmd_chat.sor.analysis.confirm import PairedCircuit
|
||||
|
||||
nopad = _arm_pairs_by_run_index(Path(data_dir), want_padding=False, bins=bins, hops=hops)
|
||||
padded = _arm_pairs_by_run_index(Path(data_dir), want_padding=True, bins=bins, hops=hops)
|
||||
return [
|
||||
PairedCircuit(nopad_pairs=tuple(nopad[i]), pad_pairs=tuple(padded[i]))
|
||||
for i in sorted(set(nopad) & set(padded))
|
||||
]
|
||||
|
||||
|
||||
def per_run_delta_aucs(paired: Sequence["PairedCircuit"]) -> List[float]:
|
||||
"""The per-run paired differences ``ΔAUC_i = AUC(no-pad,i) − AUC(+pad,i)`` — the
|
||||
auditable per-run diagnostic behind the §6 paired CI (one value per paired run
|
||||
index). Positive ⇒ padding lowered that run's linkability AUC."""
|
||||
from cmd_chat.sor.analysis.detectors import auc
|
||||
|
||||
def _auc(pairs) -> float:
|
||||
pos = [s for s, linked in pairs if linked]
|
||||
neg = [s for s, linked in pairs if not linked]
|
||||
return auc(pos, neg)
|
||||
|
||||
return [_auc(u.nopad_pairs) - _auc(u.pad_pairs) for u in paired]
|
||||
|
||||
Reference in New Issue
Block a user