diff --git a/docs/note-unique-bridge-artifact.md b/docs/note-unique-bridge-artifact.md new file mode 100644 index 0000000..3672b9d --- /dev/null +++ b/docs/note-unique-bridge-artifact.md @@ -0,0 +1,54 @@ +# Note — the RQ2-P1 "shrink" is plausibly a unique-bridge instrument artifact + +**Status:** on-record analysis note, 2026-07-21. Does **not** edit the frozen prereg or the +committed lead paper; it documents a mechanism concern for the record and motivates the +`rq2p3-mechanism-prereg.md` follow-up. Author: overseer, post-lead-paper review. + +## The finding + +The lead paper (RQ2-P1) reports federation **shrinks** the per-circuit anonymity set +(ΔH = −0.96 bits pooled; −3.63 bits bridge-federated-only, exploratory). Tracing the ratified +posterior mechanically shows this shrink is **largely forced by the instrument**, not by a +substantive funnelling phenomenon: + +1. The adversary's observation is `exit_signature = (exit_house, bridge_label)` + (`confirm_load_rq2.py:exit_signature`). +2. The anonymity set for circuit *i* is the distinct **entry** nodes among circuits sharing + *i*'s signature (`observation_consistent_sizes`): `m_i = |{entry(j) : sig(j) == sig(i)}|`. +3. The bridge-federated instrument assigns a **fresh bridge per circuit seed** + (`assembler.py:_bridge_label(seed)`), so **every circuit's `bridge_label` is unique** ⇒ + every `exit_signature` is unique ⇒ each group has size 1 ⇒ **`m_i = 1` ⇒ `H_i ≈ 0`.** + +So bridge-federated collapses to near-zero entropy **because each circuit was handed its own +bridge**, making its signature a unique identifier. That is the *same* degeneracy that made +RQ2-P3 untestable (zero-variance concentration) — here it manifests as an artificially low H +that drives ΔH negative. + +## Why this likely inverts under realistic bridge reuse + +Introduce a finite shared bridge **pool**: many circuits share a bridge ⇒ shared signature ⇒ +`m_i` = distinct entries across all of them ⇒ **larger set ⇒ higher H**. Under this posterior +a shared bridge behaves as a **mix**: more concentration plausibly *raises* anonymity — the +opposite of the naive "funnelling shrinks the set" story. The mechanism study +(`rq2p3-mechanism-prereg.md`) tests this **two-sided**; the mechanical prediction is mix +(ρ > 0), the naive prediction is funnel (ρ < 0). + +## Consistency check against observed numbers + +- Directory-federated cell (no bridge hop → `bridge_label = None`, shared signature): + observed `rq2_anonymity_entropy_bits = 5.6439 = log₂(50)` — **maximal** entropy, all 50 + senders in the set. Confirms shared signatures give large sets. +- Bridge-federated (unique bridge per circuit): H ≈ 0, consistent with the exploratory + ΔH = −3.63. The pooled −0.96 is the average of a high-H directory arm and a ≈0-H bridge arm. + +## Integrity status + +- The lead paper is **not wrong or dishonest**: §7 already flags RQ2 as conditional on the + posterior construction and RQ2-P3 as an unresolved mechanism. This note **sharpens** that + caveat from "conditional" to "the specific shrink magnitude is mechanically forced by the + fresh-bridge instrument." +- **No silent edit** to the committed paper or frozen prereg. If the mechanism study confirms + mix (ρ > 0), the honest outcome is a companion result that **qualifies/corrects** the lead + RQ2-P1 headline — reported openly, per the null-results-are-results rail. +- This strengthens the body of work: it is exactly the kind of artifact a pre-registered, + frozen-detector method is supposed to expose rather than bury. diff --git a/docs/rq2p3-mechanism-prereg.md b/docs/rq2p3-mechanism-prereg.md new file mode 100644 index 0000000..eacd2e2 --- /dev/null +++ b/docs/rq2p3-mechanism-prereg.md @@ -0,0 +1,156 @@ +# Pre-registration (DRAFT) — RQ2-P3 funnelling-mechanism study + +> **STATUS: APPROVED-PATH, PARAMS LOCKED — NOT YET FROZEN.** Design + parameters +> operator-approved 2026-07-21 (grid B∈{2,4,8}×α∈{0,1,2}; run-level cluster bootstrap; +> two-sided/direction-agnostic). Remaining step before any confirmatory cell: **human freeze +> checkpoint** — record the SHA-256 of the final text in §10 and the operator freeze sign-off. +> Build + synthetic tests + the §7 calibration gate MAY proceed now; the confirmatory battery +> MUST wait for the freeze. +> This is a **new study with its own slug** (`sor-consent-rq2p3`). It **does not modify** +> the frozen lead prereg (`sor-consent-prereg.md`, SHA +> `f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b`), whose RQ2-P3 +> result stands as reported ("inconclusive — not testable as-instrumented"). + +## 0. Why this study exists + +The lead paper found federation **shrinks** the per-circuit anonymity set (RQ2-P1, +ΔH = −0.96 bits, Holm-significant) but could **not** test the *mechanism* (RQ2-P3): the +bridge-federated instrument assigns a **fresh willing bridge per circuit seed** +(`assembler.py:131`, `_bridge_label(seed)`), so every circuit routes through a distinct +bridge, top-3 concentration is constant (`c_i = 1/C`), the covariate has **zero variance**, +and Spearman ρ is undefined (`stats.spearman` returns 0.0 by construction). The lead paper +reported this honestly as an as-instrumented degeneracy, not a null of a well-posed test. + +**The mechanical insight this study exists to test (see `note-unique-bridge-artifact.md`).** +The adversary's observation is `exit_signature = (exit_house, bridge_label)`, and the +per-circuit anonymity set `m_i` = the distinct entry nodes among circuits sharing that +signature (`confirm_load_rq2.py`). A **fresh bridge per circuit ⇒ every signature is unique +⇒ group size 1 ⇒ m_i = 1 ⇒ H ≈ 0.** That — not funnelling — is why bridge-federated produced +H ≈ 0 and drove the lead ΔH negative. Introduce a **finite shared pool** and many circuits +share a bridge ⇒ shared signature ⇒ larger set ⇒ **higher H**. Under this posterior a bridge +is a **mix**: more concentration plausibly *raises* anonymity, the opposite of the naive +funnel story. So this study is, honestly, **a test of whether the lead paper's "shrink" is a +unique-bridge artifact** — and it may *qualify or correct* the lead RQ2-P1 headline. + +This study re-instruments the willing-bridge layer with a **finite pool + skewed +willingness** so concentration genuinely varies, and — the key upgrade — treats +concentration as a **manipulated independent variable** (a dose-response design) rather than +a passive covariate. The direction is left **two-sided**: the naive prediction is funnel +(ρ < 0); the mechanical prediction under the ratified posterior is mix (ρ > 0). We pre-commit +to reporting whichever the data shows, including a correction to the lead result if warranted. +The analysis code (`bridge_concentration`, `rq2_p3_funnel`) is already correct; only the +*instrument* and this *prereg* are new. + +## 1. Blinding & integrity posture (read first) + +- **This study is NOT blind to the RQ2-P1 shrink direction** (the lead paper is published). + Mitigations that keep it honest: (a) the funnelling hypothesis was **pre-stated as a live + mechanism** in the lead paper's introduction, not invented post-hoc; (b) the confirmatory + targets below are **two-sided**; (c) concentration is a *manipulated* IV, so the test is a + designed dose-response, not a re-slice of the lead data. +- **The lead RQ2-P1 result is NOT re-litigated here.** Any ΔH this pool instrument produces + is labeled **EXPLORATORY / replication**, never a re-run of the frozen RQ2-P1. +- **No detector retuning.** The entropy estimator, posterior construction, and Spearman path + are inherited **unchanged** from the frozen lead pipeline; only bridge assignment changes. + +## 2. Research question & hypotheses + +**RQ2-P3′.** Within the consent-gated bridge-federated topology, does willing-bridge +**concentration** causally reduce the per-circuit anonymity set (funnelling)? + +- **H1 (within-cell association, two-sided).** Spearman ρ between per-circuit top-3 willing- + bridge concentration `c_i` and per-circuit entropy `H_i`. **Funnel** iff BCa 95% CI < 0; + **mix** iff CI > 0; **inconclusive** iff CI spans 0. Direction is *not* presumed. +- **H2 (dose-response, two-sided).** OLS slope β of H on realized top-3 concentration, over + **per-run mean points** (9 cells × 30 runs = 270 clustered points), cell-level BCa CI. + **Funnel** iff slope CI < 0; **mix** iff slope CI > 0. +- **H3 (joint, direction-agnostic).** Mechanism **RESOLVED** iff H1 and H2 agree in sign and + both exclude 0; the *sign* is the finding (funnel vs mix). **Unresolved** if either spans 0. + +## 3. Instrument change (the only new code) + +Replace the per-seed fresh bridge with a **willing-bridge pool** of size `B` and a fixed +skewed willingness weight vector, under a **new topology factor** so lead-paper cells stay +bit-reproducible: + +``` +# new assembler branch: topology == "bridge-federated-pool" +weights = zipf_weights(B, alpha) # fixed from the CELL seed; stable per run +idx = weighted_draw(sha256(f"sor-bridge-pool|{circuit_seed}"), weights) +bridge = f"bridge#{idx:02d}" # B distinct labels, REUSED across circuits +``` + +- Weights derive from the **cell** seed (not the circuit seed) so the willingness profile is + fixed within a run and circuits genuinely share bridges → concentration varies. +- Everything else (hop structure, houses, exit-signature grouping, posterior, Miller–Madow H, + BCa) is **identical to the frozen lead pipeline**. +- Touchpoints: `assembler.py` (new branch + pool helper), `battery.enumerate_cells()` (sweep). + `confirm_load_rq2.py`, `confirm.py`, `stats.py` are **unchanged**. + +## 4. Design matrix (concentration as IV) + +Bridge-federated-pool, selector `static`, matched-N, bridge willingness the only manipulation: + +- **Pool size** `B ∈ {2, 4, 8}` — 3 levels. +- **Willingness skew** `alpha ∈ {0 (uniform), 1.0 (moderate Zipf), 2.0 (heavy Zipf)}` — 3 levels. +- Full 3×3 = 9 concentration cells (each realizes a distinct mean top-3 concentration). + +Run order randomized within cell; deterministic from an ordering seed distinct from data seeds. + +## 5. Dependent variables + +- **Per-circuit H_i** — Miller–Madow entropy of the uniform posterior over the observation- + consistent anonymity set (inherited **verbatim** from the frozen lead pipeline). +- **Per-circuit top-3 concentration c_i** — `confirm_load_rq2.bridge_concentration` (unchanged). + +## 6. Sampling, seeds, stopping rule + +- **R = 30** seeded runs/cell, **C = 50** circuits/run (matched to the lead study). +- Base seed **S0 = 20260719**; per-cell seed = `SHA256(S0 ‖ cell_id ‖ run_index)` (matched). +- **Fixed stopping rule:** all 9 cells × R run to completion. No optional stopping, no interim + looks. An uninformative cell is reported **inconclusive**, never extended. + +## 7. Instrument-validation gate (boolean; blocks the confirmatory run) + +All must pass **before** any confirmatory cell: + +1. **Reproduce the lead degeneracy.** `B = 50, alpha = 0` → `c_i ≈ 1/50` constant, ρ undefined + — a regression check that this instrument reproduces the published RQ2-P3 result. +2. **Extremes.** `B = 1` → all circuits share one bridge → `c = 1.0`, minimal set, low H. +3. **Monotonicity.** Realized mean top-3 concentration is **monotone decreasing in B** and + **increasing in alpha**, across the sweep, on a dry (non-confirmatory) calibration pass. +4. **Entropy calibration unchanged.** H = log₂N on equiprobable synthetic senders (inherited). + +If any gate item fails → STOP, do not report; surface as NEEDS-OPERATOR. + +## 8. Analysis plan + +- Effect size + BCa 95% CI for every test; p never reported alone; 10,000 resamples; α = 0.05. +- **H1:** `confirm.rq2_p3_funnel(c, H)` per cell + pooled, but resampled with a **run-level + cluster bootstrap** (resample whole runs, not individual circuits). Rationale: circuits + sharing a bridge have identical `c_i` and correlated `H_i`, so per-circuit resampling + pseudo-replicates and falsely narrows the CI (the same defect noted for lead RQ1-P1). The + run is the independent unit. +- **H2:** OLS slope of per-run mean-H on per-run mean top-3 concentration (270 clustered + points); BCa CI resampling **over runs within cells** (cluster bootstrap). +- **Multiplicity:** Holm–Bonferroni over **this study's own family** {H1-pooled, H2-slope} + (the lead study's family-of-7 is closed and not reopened). +- **Confirmatory vs exploratory:** the 9-cell sweep is confirmatory; any per-cell ρ contrast or + ΔH replication is labeled EXPLORATORY. +- **Data exclusion (pre-data):** quarantine only on integrity failure (SHA/pcap mismatch, + non-reproducing seed). No performance-based exclusions. + +## 9. Rails (immutable, inherited) + +Prereg frozen after approval — never edited (deviations only in a stage-05-style log; no +HARKing; nulls are results). Containment: isolated docker only, self-generated fixture traffic, +lab-only. Budget $0. Worktree only on `feat/sor-consent-relay`. Raw data immutable + SHA-256. + +## 10. Freeze block (to complete at approval) + +``` +FROZEN: +PREREG SHA-256: +APPROVED BY: +POOL/ SKEW LEVELS LOCKED: B∈{2,4,8}, alpha∈{0,1.0,2.0} +``` diff --git a/docs/rq3-companion-run-brief.md b/docs/rq3-companion-run-brief.md new file mode 100644 index 0000000..330f6e1 --- /dev/null +++ b/docs/rq3-companion-run-brief.md @@ -0,0 +1,122 @@ +# Execution-freeze / run-brief (DRAFT) — RQ3 companion battery + +> **STATUS: PARAMS LOCKED 2026-07-21 — execution plan, NOT a new prereg.** RQ3's hypotheses, +> gates, and analysis are **already frozen** in `sor-consent-prereg.md` (SHA +> `f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b`, §3/§4/§6). Writing a +> "new RQ3 prereg" would be redundant and risk HARKing. This brief (a) **pins the two open +> `[APPROVAL]` execution parameters** (agent model, churn schedule — both locked below, +> blind to RQ3 outcomes) and (b) lists the build + gate steps. **GO gate before the +> confirmatory battery: the rebuild-classifier calibration gate + the churn-bites gate must +> pass green (§3-4).** Build + synthetic tests MAY proceed now. + +## 1. The frozen gates (verbatim — do not restate as new) + +Family stays the frozen size-7 {RQ1-P1, RQ1-P2, RQ2-P1, RQ2-P3, **RQ3-P1-perf, +RQ3-P1-latency, RQ3-P2**}. The three RQ3 tests were always in the family. + +| Test | Frozen gate | +|---|---| +| RQ3-P1-perf | throughput-retention(agent) − max(static,random): CI lower bound ≥ **10 pp** | +| RQ3-P1-latency | added-latency(agent): CI upper bound ≤ **100 ms** | +| RQ3-P2 | rebuild-classifier AUC: CI upper bound ≤ **0.60** | +| RQ3-P3 | logical AND: CONFIRM iff P1 ∧ P2 (not a new test) | + +R = 30 runs/cell, C = 50 circuits/run, fixed stopping rule (no optional stopping / interim +looks; uninformative cell → inconclusive). All inherited unchanged. + +## 2. Open `[APPROVAL]` params to PIN before GO (the actual gate) + +The frozen prereg left two items tagged `[APPROVAL] pinned at freeze` — **the `[APPROVAL]` +tag is still literally present in the frozen text, i.e. neither was concretely substituted.** +These must be pinned now, blind to RQ3 data: + +**(A) Agent selector model ID + weights digest** (§3 L71-73) — **LOCKED 2026-07-21.** +- Pin: **`qwen2.5:3b`** via local Ollama (`agent_selector.OllamaAgentPolicy`, temp 0, + per-run seed, state-hash cache; heuristic fallback on query failure). Record the exact + **model tag + SHA-256 weights digest** in every manifest (digest captured at run time). +- Rails satisfied: **local / open-weight, $0**, no paid-frontier call. The Claude/frontier + arm (`ClaudeExploratoryPolicy`) stays **inert / EXPLORATORY / budget-gated** — not wired. +- **Reproducibility caveat (accepted; must be stated in the paper).** Ollama at temp 0 is + **not bit-identical across machines** (quantization / GPU logit drift). The agent arm is + reproducible via the **committed decision-log + state-hash cache replay**, *not* via + independent model re-execution on other hardware — same honesty class as the RQ1 timing + caveat. Commit the decision log + cache as the reproducibility anchor. + +**(B) Churn schedule set** (§3 L74, §4) — **LOCKED 2026-07-21.** `churn.churn_schedule(seed, +nodes, steps, kill_prob_pct)` is implemented and deterministic. +- Pin: a **seeded set** of schedules — one schedule per run, seed drawn from the same + `SHA256(S0‖cell‖run)` family so every run's churn is reproducible and provenance-anchored. + **`kill_prob_pct = 30`, `steps = 20`.** +- **Churn must actually bite (gate, §4).** With these values, confirm on a dry pass that + circuits genuinely lose hops and rebuild (non-zero `drops`/`rebuilds` per run). If drop + rate is trivially zero, the throughput-retention and rebuild-classifier tests are + degenerate — STOP and surface rather than report a zero-variance null. + +## 3. Build gaps (hand to the driver in tmux; not built inline) + +Verified against the current worktree — what exists vs. what's missing: + +**Ready (no work):** `selector.py` static/random/agent all implemented + tested; +`agent_selector.OllamaAgentPolicy` reproducible; `churn.py` deterministic schedules wired +into `run_selection()`; `analysis/metrics.py` has `throughput_retention()` + +`rebuild_classifier_auc()`. + +**Missing (build):** +1. **`battery.enumerate_cells()`** (`battery.py:73`) enumerates RQ1+RQ2 only (6 cells). Add + RQ3 cells: `selector ∈ {static, random, agent} × churn-schedule(s)` at the + 1house/bridge-off control; interleave the control arm like RQ1/RQ2. +2. **`executor.run_battery()`** — wire RQ3 collection: drive `run_selection()` over the + pinned schedule; collect throughput-retention, **added-latency**, and rebuild-event gaps + → classifier AUC. **Baselines LOCKED:** added-latency = median e2e latency(agent) − + median latency(**min-latency baseline arm**, i.e. the faster of static/random); perf + margin uses **max(static, random)** per the frozen gate. +3. **Rebuild-classifier calibration gate** — calibrate the classifier on labeled + churned-vs-low-churn control signals **before** the confirmatory RQ3 battery (same + discipline as the RQ1 correlator gate). **Low-churn baseline LOCKED: `kill_prob_pct = 5`.** + Gate passes iff churned-vs-baseline is separable (AUC≈1) and baseline-vs-baseline is not + (AUC≈0.5). Boolean; blocks the run. + +## 4. Instrument-validation gate (boolean; blocks RQ3 confirmatory) + +Reuse the frozen 6-item gate spirit, plus RQ3-specific: +- Rebuild classifier calibrated on control signals (AUC≈1 churned-vs-baseline separable, + AUC≈0.5 baseline-vs-baseline) — **not fit to confirmatory cells**. +- Agent selector **reproducibility check**: same seed + state-hash → byte-identical circuit + choice (cache replay), model+digest echoed to manifest. +- Isolation (`assert engine != local`), provenance integrity (sealed `events.jsonl`). + +## 5. Analysis + +- Reuse `stage06_run` on the sealed RQ3 data; compute the three RQ3 tests with effect + BCa CI. +- **Complete the Holm step-down over the full family of 7** now that all seven p-values exist. + (The lead paper's 7/6/5/4 was a deliberately conservative *partial* embedding; the companion + computes the exact Holm-7 — both remain valid, the partial never under-corrects.) **The + companion's Holm-7 is the authoritative final correction over the whole family**, including + the lead RQ1/RQ2 tests; RQ1-P1 and RQ2-P1 survive regardless (raw p ≈ 0), but state + explicitly that the companion supersedes the lead's partial embedding. +- Confirmatory vs exploratory labeled; nulls reported honestly (a selector that does **not** + beat baselines, or a rebuild pattern that **is** a fingerprint, is the finding). + +## 6. QUIC / ssh3 transport arm — stays EXPLORATORY + +Frozen as **EXPLORATORY, orthogonal to all three RQs, and not built** (design decision D3; +§3 L75-78). Recommendation: **defer.** Promoting it doubles every cell + requires a new +transport substrate for a weak lit-gap. If ever built, carry only as a labeled EXPLORATORY +latency-sensitivity note — never in the Holm family. + +## 7. Rails (immutable, inherited) + +Frozen prereg never edited; execution pins recorded here + in manifests; no HARKing; nulls +are results. Containment: isolated docker only, self-traffic, lab-only. **Budget $0** — +local open-weight agent only, no paid arm; surface before any spend. Worktree only on +`feat/sor-consent-relay`. Raw data immutable + SHA-256. + +## 8. Pin block (to complete at GO) + +``` +GO DATE: +AGENT MODEL: qwen2.5:3b WEIGHTS DIGEST: +CHURN: kill_prob_pct=30 steps= schedules= +APPROVED BY: +FROZEN PREREG SHA (unchanged): f22331a72e0d0ccf38b787e63acabbe9d666456ec76076787a6d545c3193425b +```