paper: fill RQ3 half of combined companion (post-seal, both tracks un-blinded)
Replace the HELD-BLIND RQ3 Results/Discussion + authoritative-Holm-7
placeholders in docs/stage-07-companion-methods.md with real numbers read
from the sealed record only (rq3-confirmatory-analysis.json e09c66ef…):
RQ3-P1-perf -0.6pp CI[-1.58,+0.39]pp -> H0 (no +10pp agent margin)
RQ3-P1-latency -13.5ms CI[-52.1,+34.9]ms -> within 100ms budget
RQ3-P2 AUC 0.587 CI[0.458,0.703] -> fingerprint NOT excluded
RQ3-P3 = H0 (perf fails AND P2 fails)
Authoritative Holm-7 (family size 7, supersedes lead partial embedding, D3):
survivors RQ1-P1 / RQ2-P1 / RQ2-P3; non-survivors RQ1-P2 / RQ3-P2 /
RQ3-P1-perf / RQ3-P1-latency. RQ2-P3 slot carries the mechanism-corrected
primary p, not the lead degenerate p=1.
Calibration disclosure carried: the 0.93 RQ3 calibration AUC is regime
discrimination (churn kp30 vs kp5), distinct from the 0.587 confirmatory
selector-vs-selector fingerprint — no HARKing, detectors frozen.
RQ2-P3 half not re-litigated (kept as filled at e0d865d). Nulls reported
openly as results. Frozen lead prereg f22331a72e… and RQ2-P3 prereg
8db4e8a7… both intact. $0/offline, worktree-only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -109,3 +109,4 @@ RQ2 RATIFIED (operator Andre, 2026-07-20, while blind) — observation-consisten
|
||||
- 2026-07-22 RQ3 BATTERY COMPLETE (90/90 runs, live-docker-e2e, exited clean) → UN-BLIND + FROZEN analysis. RAW SEALED: `.../20260722T040640Z/confirmatory-data/SHA256SUMS` (battery-results.json `5b61e461…` + 90 rq3-run.json sidecars). New analyzer `analysis/rq3_confirm.py` (RQ3 analogue of rq2p3_confirm; frozen `stats`/`metrics` UNTOUCHED — run-level multi-arm bootstrap mirrors `two_sample_diff_ci`, 10k BCa, α=0.05). Sealed analysis `.../20260722T040640Z/analysis/rq3-confirmatory-analysis.json` (SHA `e09c66ef…`). Tests `tests/test_sor_rq3_confirm.py` 6 passed; full SOR suite 207 passed (no regression).
|
||||
- RQ3 RESULT (effect+CI, never bare p): **RQ3-P1-perf FAIL/H0** — retention margin agent−max(static,random) = **−0.6pp**, BCa CI [−1.58pp, +0.39pp]; every selector heals ~all churn drops (~99% retention) so no ≥+10pp agent gain. **RQ3-P1-latency HOLDS budget** — added-latency(agent−min-baseline=random) = **−13.5ms**, CI [−52.1, +34.9]ms, upper ≤100ms (agent not slower). **RQ3-P2 FAIL/not-excluded** — rebuild-classifier AUC(agent vs pooled baseline, per-run mean-gap) = **0.587**, CI [0.458, 0.703], upper 0.703 > 0.60 → fingerprint NOT excluded (underpowered at n=30). **RQ3-P3 = H0** (P1-perf fails ∧ P2 fails). Honest null: the local agent selector neither beats baselines nor is certifiably non-classifiable.
|
||||
- AUTHORITATIVE HOLM-7 (frozen family size=7; supersedes lead's conservative partial embedding, D3). RQ2-P3 slot carries the mechanism-corrected primary **H1-pooled Spearman p=0** (not the lead's degenerate p=1). **SURVIVORS: RQ1-P1 (r1×7), RQ2-P1 shrink (r2×6), RQ2-P3 mix (r3×5)** all holm_p=0. NON-survivors: RQ1-P2 (r4×4, holm_p=0.365), RQ3-P2 (r5×3, 0.511), RQ3-P1-perf (r6×2, 0.511), RQ3-P1-latency (r7×1, 0.511). Lead RQ1-P1 & RQ2-P1 survive regardless. Both prereg SHAs intact; $0/offline; worktree-only. NEXT: fill RQ3 half of companion paper (stone 2).
|
||||
- 2026-07-22 COMPANION PAPER — RQ3 HALF FILLED POST-SEAL (stone 2, D3 combined paper). `docs/stage-07-companion-methods.md`: §5 Results + §6 Discussion RQ3 placeholders AND the authoritative-Holm-7 placeholder REPLACED with real numbers FROM THE SEALED RECORD ONLY (perf −0.6pp CI[−1.58,+0.39]pp H0; latency −13.5ms CI[−52.1,+34.9] within budget, min-baseline=random; P2 AUC 0.587 CI[0.458,0.703] fingerprint-not-excluded; P3=H0). Authoritative Holm-7 in-paper: survivors RQ1-P1/RQ2-P1/RQ2-P3, non-survivors RQ1-P2/RQ3-P2/RQ3-P1-perf/RQ3-P1-latency. Abstract + top blinding block un-blinded (both tracks). RQ2-P3 half NOT re-litigated (kept as filled at e0d865d). DISCLOSURE carried: 0.93 RQ3 calibration AUC = regime discrimination (churn kp30 vs kp5), DISTINCT from the 0.587 confirmatory selector-vs-selector fingerprint — no HARKing, gates frozen. $0/offline, worktree-only; both prereg SHAs intact. NEXT: report + HOLD for review.
|
||||
|
||||
Reference in New Issue
Block a user