SS3(RQ1-P2): freeze-derived run_index paired bootstrap + loader wiring
Pairing ruling (docs/stage-05-rq1p2-pairing-clarification.md, RATIFIED-by- derivation, made while BLIND): the frozen §6 L197-198 "paired" bootstrap pairs the bridge-on (no-pad) and bridge-on+padding arms BY RUN INDEX — the only balanced pairing the R=30 (§4 L111) interleaved (§3 L97) design supports; the §4 L108 RQ1 unit is AUC over the (entry,exit) pair set, so the pairable unit is the run, not the circuit. Prereg SHA untouched (f22331a72e…). confirm_load.collect_rq1_p2_paired feeds one confirm.PairedCircuit per shared run_index into the FROZEN confirm.rq1_p2_padding (§6 arm-level ΔAUC preserved); per_run_delta_aucs exposes the auditable per-run ΔAUC_i; missing-arm indices are unpaired (frozen R not redefined). Unit-tested on SYNTHETIC pcaps only (prereg §2 blinding — unblocks CODE not RESULTS); full SOR suite 176 passed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -49,3 +49,7 @@ RQ2 RATIFIED (operator Andre, 2026-07-20, while blind) — observation-consisten
|
|||||||
- DRIVE STATE: SS1 ✅ SS2 firing ✅ SS3-RQ1 ✅ RQ2-proposal ✅ SS4-skeleton ✅. Battery: 492/9000, RUNNING healthy (PID 2539198). REMAINING GATES: SS3-RQ2 + RQ1-P2 pairing on Andre's RQ2 ratification; SS4 Results/Discussion + SS5 review on (a) battery completion (blinding) AND (b) RQ2 ratification. All data-independent overnight work now exhausted without fabricating or breaking blinding.
|
- DRIVE STATE: SS1 ✅ SS2 firing ✅ SS3-RQ1 ✅ RQ2-proposal ✅ SS4-skeleton ✅. Battery: 492/9000, RUNNING healthy (PID 2539198). REMAINING GATES: SS3-RQ2 + RQ1-P2 pairing on Andre's RQ2 ratification; SS4 Results/Discussion + SS5 review on (a) battery completion (blinding) AND (b) RQ2 ratification. All data-independent overnight work now exhausted without fabricating or breaking blinding.
|
||||||
- 2026-07-20 RQ2 RATIFIED + offline loader DONE + committed: docs/stage-05-rq2-posterior-clarification.md flipped PROPOSED→RATIFIED (operator Andre, while BLIND — 0 RQ2 stats computed). Built cmd_chat/sor/analysis/confirm_load_rq2.py: reconstructs each circuit's spec OFFLINE from per_circuit_seeds via deterministic assemble(), derives the observation-consistent anonymity set A_i (uniform/max-entropy → [1]*m_i), per-circuit H_i=Miller–Madow, and the willing-bridge concentration series — feeding confirm.rq2_p1_delta_h (ΔH two-sided) + rq2_p3_funnel (Spearman). Unit-tested on SYNTHETIC specs/seeds ONLY (test_sor_confirm_load_rq2.py, 6 passed; full SOR suite 172 passed). NEEDS-OPERATOR banner removed (resolved).
|
- 2026-07-20 RQ2 RATIFIED + offline loader DONE + committed: docs/stage-05-rq2-posterior-clarification.md flipped PROPOSED→RATIFIED (operator Andre, while BLIND — 0 RQ2 stats computed). Built cmd_chat/sor/analysis/confirm_load_rq2.py: reconstructs each circuit's spec OFFLINE from per_circuit_seeds via deterministic assemble(), derives the observation-consistent anonymity set A_i (uniform/max-entropy → [1]*m_i), per-circuit H_i=Miller–Madow, and the willing-bridge concentration series — feeding confirm.rq2_p1_delta_h (ΔH two-sided) + rq2_p3_funnel (Spearman). Unit-tested on SYNTHETIC specs/seeds ONLY (test_sor_confirm_load_rq2.py, 6 passed; full SOR suite 172 passed). NEEDS-OPERATOR banner removed (resolved).
|
||||||
- HARD BLIND HOLD STILL ON: collect_rq2_p1_arms/collect_rq2_p3 are BLIND-GATED — NOT run on real data; no RQ2-P1/P2/P3 result computed/inspected/reported until battery completes. Battery: 3816/9000 (~42%), alive, healthy. INSTRUMENT CAVEAT for SS5: bridge-federated assigns a fresh bridge per circuit seed ⇒ willing-bridge reuse is minimal ⇒ P3 concentration may be near-degenerate as-instrumented (an honest, reportable property — loader does not fix it).
|
- HARD BLIND HOLD STILL ON: collect_rq2_p1_arms/collect_rq2_p3 are BLIND-GATED — NOT run on real data; no RQ2-P1/P2/P3 result computed/inspected/reported until battery completes. Battery: 3816/9000 (~42%), alive, healthy. INSTRUMENT CAVEAT for SS5: bridge-federated assigns a fresh bridge per circuit seed ⇒ willing-bridge reuse is minimal ⇒ P3 concentration may be near-degenerate as-instrumented (an honest, reportable property — loader does not fix it).
|
||||||
|
- 2026-07-20 RQ1-P2 PAIRING RESOLVED = by run_index (overseer ruling, freeze-DERIVED — the Holm case, not the RQ2 gap-fill). docs/stage-05-rq1p2-pairing-clarification.md drafted: cites frozen §6 L197-198 (paired mandated), §4 L108 (RQ1 unit=AUC over the pair set ⇒ pairable unit is the RUN not the circuit), §4 L111 (R=30 balanced), §3 L97 (interleaving indexes shared session position); status RATIFIED-by-derivation / PROPOSED for Andre (low-risk, non-blocking); prereg UNTOUCHED.
|
||||||
|
- Loader wired: confirm_load.collect_rq1_p2_paired maps each run_index present in BOTH bridge=on (no-pad) & bridge=on+padding arms → one confirm.PairedCircuit (its two arms' pair sets), fed to the FROZEN confirm.rq1_p2_padding (§6 arm-level ΔAUC preserved; decision layer untouched); per_run_delta_aucs exposes the auditable per-run ΔAUC_i diagnostic; missing-arm run indices are simply unpaired (frozen R NOT redefined; inconclusive if too few remain).
|
||||||
|
- Unit-tested SYNTHETIC pcaps ONLY (test_sor_confirm_load_rq1p2.py, 4 passed: pair-by-run-index, positive ΔAUC_i when padding flattens, feeds frozen rq1_p2_padding CI>0, graceful missing-arm); full SOR suite 176 passed. No confirmatory record read (prereg §2 blinding: unblocks CODE not RESULTS).
|
||||||
|
- HARD BLIND HOLD REMAINS: no real RQ1-P2 number until battery completes (battery ~78/180 runs ≈ 43%, alive PID 2539198). prereg SHA INTACT (f22331a72e…); worktree-only feat/sor-consent-relay.
|
||||||
|
|||||||
@@ -8,12 +8,15 @@ The candidate true pairing is the diagonal (same circuit); off-diagonal pairs ar
|
|||||||
unlinked — exactly the §4 "AUC over the (entry, exit) pair set, bootstrap over
|
unlinked — exactly the §4 "AUC over the (entry, exit) pair set, bootstrap over
|
||||||
circuit pairs" unit.
|
circuit pairs" unit.
|
||||||
|
|
||||||
Deliberately NOT here (held for the operator ruling — see OVERSEER NEEDS-OPERATOR):
|
RQ1-P2 (padding efficacy) is wired here too: the frozen §6 **paired** bootstrap
|
||||||
* **RQ2** — the frozen DV is a *per-circuit adversary sender posterior* (§3/§4),
|
pairs the bridge-on (no-pad) and bridge-on+padding arms **by run index** — the only
|
||||||
whose construction is not formula-pinned in the prereg and not produced by the
|
balanced pairing the R=30 interleaved design supports (freeze-derived; see
|
||||||
instrument; guessing a posterior model post-freeze would be HARKing.
|
`docs/stage-05-rq1p2-pairing-clarification.md`). Each run index becomes one
|
||||||
* **RQ1-P2** nopad/pad *pairing* convention for the paired bootstrap — the pairing
|
`confirm.PairedCircuit` fed to the frozen `confirm.rq1_p2_padding`.
|
||||||
rule across the bridge-on and bridge-on+padding arms is a choice; not guessed.
|
|
||||||
|
Split out (RQ2 lives in its own loader): **RQ2** — the per-circuit adversary
|
||||||
|
sender posterior (§3/§4) is reconstructed in `confirm_load_rq2.py` per the RATIFIED
|
||||||
|
stage-05 posterior clarification.
|
||||||
|
|
||||||
Blinding (prereg §2, binding): callers must **not** run this on a confirmatory
|
Blinding (prereg §2, binding): callers must **not** run this on a confirmatory
|
||||||
data dir until the full battery has completed — no inspection of intermediate
|
data dir until the full battery has completed — no inspection of intermediate
|
||||||
@@ -24,7 +27,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Dict, List, Tuple
|
from typing import Dict, List, Sequence, Tuple
|
||||||
|
|
||||||
from cmd_chat.sor.analysis.detectors import score_matrix
|
from cmd_chat.sor.analysis.detectors import score_matrix
|
||||||
from cmd_chat.sor.executor import DEFAULT_BINS, ExecutorError, bin_pcap_bytes
|
from cmd_chat.sor.executor import DEFAULT_BINS, ExecutorError, bin_pcap_bytes
|
||||||
@@ -43,6 +46,7 @@ def classify_run(run_dir: Path) -> Dict[str, object]:
|
|||||||
"cell_id": d["cell_id"],
|
"cell_id": d["cell_id"],
|
||||||
"rq": d["rq"],
|
"rq": d["rq"],
|
||||||
"padding_applied": bool(d.get("padding_applied", False)),
|
"padding_applied": bool(d.get("padding_applied", False)),
|
||||||
|
"run_index": d.get("run_index"), # needed for the §6 RQ1-P2 run-index pairing
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -88,3 +92,63 @@ def collect_rq1_p1_pairs(data_dir: Path, *, bins: int = DEFAULT_BINS, hops: int
|
|||||||
if info["rq"] == "RQ1" and str(info["cell_id"]).endswith("bridge=on") and not info["padding_applied"]:
|
if info["rq"] == "RQ1" and str(info["cell_id"]).endswith("bridge=on") and not info["padding_applied"]:
|
||||||
pairs.extend(reconstruct_run_pairs(run_dir, bins=bins, hops=hops))
|
pairs.extend(reconstruct_run_pairs(run_dir, bins=bins, hops=hops))
|
||||||
return pairs
|
return pairs
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# RQ1-P2 (padding efficacy) — the frozen §6 PAIRED bootstrap, paired BY RUN INDEX.
|
||||||
|
# Freeze-derived pairing (docs/stage-05-rq1p2-pairing-clarification.md): each run
|
||||||
|
# index i yields one confirm.PairedCircuit carrying that run's no-pad and +pad pair
|
||||||
|
# sets, fed to the frozen confirm.rq1_p2_padding. BLIND-GATED on real data.
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
def _arm_pairs_by_run_index(
|
||||||
|
data_dir: Path, *, want_padding: bool, bins: int, hops: int
|
||||||
|
) -> Dict[int, List[Pair]]:
|
||||||
|
"""Map run_index -> that run's RQ1 (entry,exit) pair set, for the bridge-on arm
|
||||||
|
with (``want_padding``) or without padding. Runs missing a run_index in their
|
||||||
|
persisted metrics.json are skipped (never guessed)."""
|
||||||
|
out: Dict[int, List[Pair]] = {}
|
||||||
|
for run_dir in sorted(p for p in Path(data_dir).iterdir() if p.is_dir()):
|
||||||
|
if not (run_dir / "metrics.json").exists():
|
||||||
|
continue
|
||||||
|
info = classify_run(run_dir)
|
||||||
|
if info["rq"] != "RQ1" or not str(info["cell_id"]).endswith(
|
||||||
|
"bridge=on+padding" if want_padding else "bridge=on"
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
if bool(info["padding_applied"]) != want_padding or info["run_index"] is None:
|
||||||
|
continue
|
||||||
|
out[int(info["run_index"])] = reconstruct_run_pairs(run_dir, bins=bins, hops=hops)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def collect_rq1_p2_paired(
|
||||||
|
data_dir: Path, *, bins: int = DEFAULT_BINS, hops: int = 3
|
||||||
|
) -> List["PairedCircuit"]:
|
||||||
|
"""The RQ1-P2 paired units: one :class:`confirm.PairedCircuit` per run index
|
||||||
|
present in **both** the bridge-on (no-pad) and bridge-on+padding arms — the
|
||||||
|
freeze-derived run-index pairing (§6 "paired bootstrap"). Graceful: a run index
|
||||||
|
missing an arm is simply not paired (the frozen R is not silently redefined);
|
||||||
|
the caller reports inconclusive if too few paired units remain. BLIND-GATED —
|
||||||
|
do not call on a confirmatory dir until the battery completes (prereg §2)."""
|
||||||
|
from cmd_chat.sor.analysis.confirm import PairedCircuit
|
||||||
|
|
||||||
|
nopad = _arm_pairs_by_run_index(Path(data_dir), want_padding=False, bins=bins, hops=hops)
|
||||||
|
padded = _arm_pairs_by_run_index(Path(data_dir), want_padding=True, bins=bins, hops=hops)
|
||||||
|
return [
|
||||||
|
PairedCircuit(nopad_pairs=tuple(nopad[i]), pad_pairs=tuple(padded[i]))
|
||||||
|
for i in sorted(set(nopad) & set(padded))
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def per_run_delta_aucs(paired: Sequence["PairedCircuit"]) -> List[float]:
|
||||||
|
"""The per-run paired differences ``ΔAUC_i = AUC(no-pad,i) − AUC(+pad,i)`` — the
|
||||||
|
auditable per-run diagnostic behind the §6 paired CI (one value per paired run
|
||||||
|
index). Positive ⇒ padding lowered that run's linkability AUC."""
|
||||||
|
from cmd_chat.sor.analysis.detectors import auc
|
||||||
|
|
||||||
|
def _auc(pairs) -> float:
|
||||||
|
pos = [s for s, linked in pairs if linked]
|
||||||
|
neg = [s for s, linked in pairs if not linked]
|
||||||
|
return auc(pos, neg)
|
||||||
|
|
||||||
|
return [_auc(u.nopad_pairs) - _auc(u.pad_pairs) for u in paired]
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Stage-05 analysis clarification — RQ1-P2 paired-bootstrap pairing = by run_index
|
||||||
|
|
||||||
|
**Status: RATIFIED-by-derivation, 2026-07-20 — PROPOSED for operator (Andre)
|
||||||
|
confirmation (low-risk, freeze-faithful, non-blocking).** Worktree clarification
|
||||||
|
note only — **NOT a deviation**. This *reads an implied unit out of* the frozen
|
||||||
|
prereg; it does **not** edit the frozen prereg (`sor-consent-prereg.md`, SHA
|
||||||
|
`f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b`).
|
||||||
|
|
||||||
|
**Made while BLIND** (battery mid-collection, **0 RQ1 confirmatory statistics
|
||||||
|
computed, inspected, or reported**). Like the Holm clarification — and *unlike* the
|
||||||
|
RQ2-posterior clarification — this adds **no construct**: it restates the pairing
|
||||||
|
the frozen text already implies for the §6 *paired* bootstrap.
|
||||||
|
|
||||||
|
> **Distinction from the RQ2 case.** The RQ2 note filled a genuine gap (the prereg
|
||||||
|
> pinned a per-circuit posterior but no construction rule → it needed operator
|
||||||
|
> *ratification*). This note is the **Holm case**: the frozen text fixes the
|
||||||
|
> answer; naming it is compliance, not a new methodological choice. Hence
|
||||||
|
> "RATIFIED-by-derivation" — operator confirmation is a courtesy check, not a gate.
|
||||||
|
|
||||||
|
## The question
|
||||||
|
|
||||||
|
Frozen §6 RQ1-P2 mandates a **paired** bootstrap of a padding-efficacy difference:
|
||||||
|
|
||||||
|
- **§6 (Analysis), RQ1-P2** (prereg L197–198): *"ΔAUC = AUC(bridge-on, no-pad) −
|
||||||
|
AUC(bridge-on, +pad); **paired** bootstrap 95% CI. Padding effective iff ΔAUC CI
|
||||||
|
**> 0**."*
|
||||||
|
|
||||||
|
"Paired" is **mandated** — an unpaired / two-sample bootstrap is out. But a single
|
||||||
|
physical circuit is run under **either** the no-pad **or** the +pad arm, never both
|
||||||
|
(they are different §2 cells with different `cell_id`s → different per-run seeds).
|
||||||
|
So: **what is the paired unit?** Earlier this was held as a "choice" on the grounds
|
||||||
|
that per-arm *seeds differ so nothing pairs* — but that conflates **seed-pairing**
|
||||||
|
with **run-pairing**.
|
||||||
|
|
||||||
|
## Authority (the freeze fixes the unit)
|
||||||
|
|
||||||
|
Three frozen facts jointly determine the pairing:
|
||||||
|
|
||||||
|
- **§4 unit of analysis** (prereg L108): *"RQ1 — the (entry, exit) circuit-segment
|
||||||
|
pair (**AUC over the pair set**)."* The RQ1 statistic is an **arm/cell-level set
|
||||||
|
AUC**, *not* a per-circuit quantity. So the pairable unit is never the individual
|
||||||
|
circuit — it is the thing that yields one AUC.
|
||||||
|
- **§4 repetition** (prereg L111): *"**R = 30** independent seeded runs per design
|
||||||
|
cell."* Both the no-pad and the +pad arms have the **same balanced R = 30** runs,
|
||||||
|
indexed `i ∈ {0..29}`. This is the only structure that is **1-to-1 across the two
|
||||||
|
arms**.
|
||||||
|
- **§3 confounds L97 / executor design** (interleaving): run order is *randomized
|
||||||
|
and controls interleaved* to bracket **per-session thermal / background-load
|
||||||
|
drift** on the grid. Run index `i` therefore indexes a **shared replicate /
|
||||||
|
session position** whose nuisance the pairing is meant to remove.
|
||||||
|
|
||||||
|
Pairing by run index is the **only balanced pairing the R = 30 interleaved design
|
||||||
|
supports**, and it is exactly the nuisance-removal a *paired* bootstrap exists for.
|
||||||
|
|
||||||
|
## Decision (freeze-derived, implemented)
|
||||||
|
|
||||||
|
**RQ1-P2 pairs by run index `i ∈ {0..29}`.** For each run index `i`:
|
||||||
|
|
||||||
|
1. `AUC_nopad(i)` = the correlator AUC over run `i`'s **(entry, exit) pair set** in
|
||||||
|
the **bridge=on** (no-pad) cell;
|
||||||
|
2. `AUC_pad(i)` = the AUC over run `i`'s pair set in the **bridge=on+padding** cell;
|
||||||
|
3. `ΔAUC_i = AUC_nopad(i) − AUC_pad(i)` — one paired difference per run index.
|
||||||
|
|
||||||
|
The **paired bootstrap 95% CI** resamples the **run-index paired unit** (each unit
|
||||||
|
carries both arms' pair sets for that `i`); the reported effect size is the §6
|
||||||
|
arm-level `ΔAUC = AUC(no-pad) − AUC(+pad)` and the gate is **unchanged: padding
|
||||||
|
effective iff the CI > 0.** Realized as one `confirm.PairedCircuit` per run index
|
||||||
|
fed to the frozen `confirm.rq1_p2_padding` decision function (the §6 decision layer
|
||||||
|
is **not** modified) — the loader only shapes the paired units.
|
||||||
|
|
||||||
|
**Validity note.** Pairing removes the shared replicate/session nuisance under
|
||||||
|
interleaving; it never invalidates the CI. If replicate correlation happens to be
|
||||||
|
0, the paired bootstrap reduces to ~unpaired conservativeness — so pairing is
|
||||||
|
weakly dominant and safe. Differing per-arm *seeds* are irrelevant: the pairing is
|
||||||
|
over the **run position**, not the seed.
|
||||||
|
|
||||||
|
**Graceful degradation (no R-dropping).** All R = 30 run indices are attempted; a
|
||||||
|
run index missing one arm (e.g. an integrity-quarantined run, §6 exclusion rule) is
|
||||||
|
simply not paired — the frozen R is **not** silently redefined, and if too few
|
||||||
|
paired units remain the test reports **inconclusive** (never a shrink-to-fit).
|
||||||
|
|
||||||
|
*Covered by `tests/test_sor_confirm_load_rq1p2.py` (SYNTHETIC pcaps only): 30
|
||||||
|
paired `ΔAUC_i` from per-run AUCs, paired-bootstrap CI via `confirm.rq1_p2_padding`,
|
||||||
|
and graceful pairing when a run is missing an arm.*
|
||||||
|
|
||||||
|
## Blinding (binding)
|
||||||
|
|
||||||
|
Ratification-by-derivation unblocks the **CODE, not the results** (prereg §2). The
|
||||||
|
`collect_rq1_p2_paired` real-data entrypoint is **BLIND-GATED** — it must not run on
|
||||||
|
a confirmatory data dir until the full battery **completes** (`battery-results.json`
|
||||||
|
present, or all 180 cell-runs carry `metrics.json`). The pairing logic is
|
||||||
|
unit-tested on **synthetic** pcaps only, mirroring the RQ1-P1 loader (`8224a38`) and
|
||||||
|
the RQ2 loader (`bd7e376`).
|
||||||
|
|
||||||
|
## Operator confirmation (requested, non-blocking)
|
||||||
|
|
||||||
|
Andre: this is a **freeze-derived** reading (the Holm pattern), not a new construct.
|
||||||
|
Confirm at leisure; no RQ1-P2 number is computed until the battery completes
|
||||||
|
regardless. If you would rather pair differently (e.g. pool to a single arm-level
|
||||||
|
ΔAUC with an unpaired CI), that would be a **deviation** (§8) since §6 says
|
||||||
|
*paired* — flag it and it goes in `sor-consent-deviations.md`.
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
"""SS3 RQ1-P2 loader — the freeze-DERIVED run-index paired bootstrap, on SYNTHETIC
|
||||||
|
pcaps only (docs/stage-05-rq1p2-pairing-clarification.md).
|
||||||
|
|
||||||
|
These pin that the loader pairs the bridge-on (no-pad) and bridge-on+padding arms
|
||||||
|
**by run index** into one `confirm.PairedCircuit` per index — the only balanced
|
||||||
|
pairing the R=30 interleaved design supports — and that the frozen §6 decision
|
||||||
|
function `confirm.rq1_p2_padding` consumes those units unchanged. NO confirmatory
|
||||||
|
record is read (prereg §2 blinding): run dirs are hand-built with synthetic pcaps
|
||||||
|
and metrics.json that carry a `run_index`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import struct
|
||||||
|
|
||||||
|
from cmd_chat.sor.analysis import confirm, confirm_load
|
||||||
|
|
||||||
|
|
||||||
|
def _write_series_pcap(path, series):
|
||||||
|
"""Minimal LINKTYPE_RAW pcap placing one packet of size series[k] at t=k, so
|
||||||
|
bin_pcap_bytes(path, bins=len(series)) reproduces `series` bin-for-bin."""
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(path, "wb") as f:
|
||||||
|
f.write(struct.pack("<IHHiIII", 0xA1B2C3D4, 2, 4, 0, 0, 65535, 101))
|
||||||
|
for k, nbytes in enumerate(series):
|
||||||
|
f.write(struct.pack("<IIII", k, 0, nbytes, nbytes))
|
||||||
|
f.write(b"x" * nbytes)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_circuit(run_dir, cid, ingress_series, egress_series):
|
||||||
|
pdir = run_dir / "circuits" / cid / "pcap"
|
||||||
|
_write_series_pcap(pdir / "hop0.pcap", ingress_series) # ingress
|
||||||
|
_write_series_pcap(pdir / "hop2.pcap", egress_series) # egress (hops=3)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_metrics(run_dir, cell_id, rq, padding, run_index):
|
||||||
|
"""Like the RQ1-P1 helper but ALSO persists run_index — the pairing key."""
|
||||||
|
run_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
(run_dir / "metrics.json").write_text(
|
||||||
|
json.dumps({
|
||||||
|
"cell_id": cell_id, "rq": rq,
|
||||||
|
"padding_applied": padding, "run_index": run_index,
|
||||||
|
}),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_ON = "RQ1/topo=1house/selector=static/bridge=on"
|
||||||
|
_PAD = "RQ1/topo=1house/selector=static/bridge=on+padding"
|
||||||
|
|
||||||
|
|
||||||
|
def _write_arm_run(tmp_path, *, run_index, padding):
|
||||||
|
"""A one-run arm dir: two circuits, each hop0/hop2 sharing that circuit's own
|
||||||
|
temporal profile so the diagonal (linked) pearson beats the cross (unlinked).
|
||||||
|
The +pad arm is flattened so its measured linkability is lower than no-pad."""
|
||||||
|
cell = _PAD if padding else _ON
|
||||||
|
rd = tmp_path / f"cell-{'pad' if padding else 'on'}-r{run_index}"
|
||||||
|
_write_metrics(rd, cell, "RQ1", padding, run_index)
|
||||||
|
if padding:
|
||||||
|
# Padding masks the temporal profile → both circuits look flat → low AUC.
|
||||||
|
_write_circuit(rd, "c0", [50, 50, 50, 50], [50, 50, 50, 50])
|
||||||
|
_write_circuit(rd, "c1", [50, 50, 50, 50], [50, 50, 50, 50])
|
||||||
|
else:
|
||||||
|
_write_circuit(rd, "c0", [100, 10, 100, 10], [100, 10, 100, 10])
|
||||||
|
_write_circuit(rd, "c1", [10, 100, 10, 100], [10, 100, 10, 100])
|
||||||
|
return rd
|
||||||
|
|
||||||
|
|
||||||
|
def test_pairs_bridge_on_arms_by_run_index(tmp_path):
|
||||||
|
# Three run indices present in BOTH arms → three PairedCircuit units.
|
||||||
|
for i in range(3):
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=False)
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=True)
|
||||||
|
paired = confirm_load.collect_rq1_p2_paired(tmp_path, bins=4, hops=3)
|
||||||
|
assert len(paired) == 3
|
||||||
|
# Each unit carries both arms' pair sets (2x2 score matrix → 4 pairs each).
|
||||||
|
for u in paired:
|
||||||
|
assert len(u.nopad_pairs) == 4 and len(u.pad_pairs) == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_per_run_delta_aucs_are_positive_when_padding_lowers_linkability(tmp_path):
|
||||||
|
for i in range(4):
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=False)
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=True)
|
||||||
|
paired = confirm_load.collect_rq1_p2_paired(tmp_path, bins=4, hops=3)
|
||||||
|
deltas = confirm_load.per_run_delta_aucs(paired)
|
||||||
|
assert len(deltas) == 4 # one ΔAUC_i per paired run index
|
||||||
|
# no-pad AUC ~1 (linked beats cross), +pad AUC ~0.5 (flat) → ΔAUC_i > 0.
|
||||||
|
assert all(d > 0.0 for d in deltas)
|
||||||
|
|
||||||
|
|
||||||
|
def test_feeds_frozen_rq1_p2_padding_decision(tmp_path):
|
||||||
|
for i in range(5):
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=False)
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=True)
|
||||||
|
paired = confirm_load.collect_rq1_p2_paired(tmp_path, bins=4, hops=3)
|
||||||
|
test = confirm.rq1_p2_padding(paired, seed=1, n_resamples=200)
|
||||||
|
assert test.name == "RQ1-P2" and test.effect == "ΔAUC"
|
||||||
|
assert test.ci.point > 0.0 and test.decision == "padding-effective"
|
||||||
|
|
||||||
|
|
||||||
|
def test_graceful_when_a_run_index_missing_an_arm(tmp_path):
|
||||||
|
# run_index 0,1 in both arms; run_index 2 only no-pad, run_index 3 only +pad.
|
||||||
|
for i in (0, 1):
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=False)
|
||||||
|
_write_arm_run(tmp_path, run_index=i, padding=True)
|
||||||
|
_write_arm_run(tmp_path, run_index=2, padding=False) # unpaired
|
||||||
|
_write_arm_run(tmp_path, run_index=3, padding=True) # unpaired
|
||||||
|
paired = confirm_load.collect_rq1_p2_paired(tmp_path, bins=4, hops=3)
|
||||||
|
# Only the two indices present in BOTH arms are paired; frozen R is not
|
||||||
|
# silently redefined — the unpaired runs are simply dropped from the pairing.
|
||||||
|
assert len(paired) == 2
|
||||||
Reference in New Issue
Block a user