review: stage-08 adversarial red-team of the combined companion paper (PASS)

3-persona integrity red-team of docs/stage-07-companion-methods.md (RQ2-P3
mechanism + RQ3), same discipline as the lead stage-08 review. Runs only on
already-sealed artifacts; computes no new statistic; edits no frozen prereg.

METHODOLOGIST: reproduced the authoritative Holm-7 step-down independently,
byte-for-byte vs the sealed record (survivors RQ1-P1 / RQ2-P1 / RQ2-P3;
non-survivors RQ1-P2 / RQ3-P2 / RQ3-P1-perf / RQ3-P1-latency). Confirmed the
supersedes-partial-embedding claim, the single-slot RQ2-P3 mechanism-corrected
p (family_size 7), run-level cluster-bootstrap validity, two-sided pre-commit.

DOMAIN SKEPTIC: mix-qualifies-shrink scoped as correction not overwrite;
calibration preview (dry-pass rho -> +0.838) disclosed and kept DISTINCT from
the confirmatory pooled +0.6244; RQ3 double-null unspun (perf = retention
ceiling / no headroom; P2 = n=30 power limitation, not a false all-clear).

REPRODUCIBILITY AUDITOR: every figure matches the sealed records (rq2p3
5fdcb379, rq3 analysis e09c66ef, rq3 battery 5b61e461); the 0.93 calibration
AUC (kp30-vs-kp5 regime) kept distinct from the 0.587 confirmatory
agent-vs-baseline AUC (no HARKing); Ollama temp-0 cross-hardware caveat
present; both prereg SHAs (f22331a72e, 8db4e8a7) recomputed intact; blinding
integrity = each track filled once, post-seal.

One prose-clarity defect found and fixed: §5 Holm-7 ranks 6-7 report the
step-down monotone-enforced 0.511 (bare products 0.354 / 0.497); added a
one-clause note so it cannot read as an arithmetic slip. ZERO numbers changed.

Worktree-only on feat/sor-consent-relay; containment intact; $0/offline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
leetcrypt
2026-07-22 09:15:22 -07:00
parent 1b6449b827
commit 7b28e0a89d
3 changed files with 203 additions and 1 deletions
+1
View File
@@ -110,3 +110,4 @@ RQ2 RATIFIED (operator Andre, 2026-07-20, while blind) — observation-consisten
- RQ3 RESULT (effect+CI, never bare p): **RQ3-P1-perf FAIL/H0** — retention margin agentmax(static,random) = **0.6pp**, BCa CI [1.58pp, +0.39pp]; every selector heals ~all churn drops (~99% retention) so no ≥+10pp agent gain. **RQ3-P1-latency HOLDS budget** — added-latency(agentmin-baseline=random) = **13.5ms**, CI [52.1, +34.9]ms, upper ≤100ms (agent not slower). **RQ3-P2 FAIL/not-excluded** — rebuild-classifier AUC(agent vs pooled baseline, per-run mean-gap) = **0.587**, CI [0.458, 0.703], upper 0.703 > 0.60 → fingerprint NOT excluded (underpowered at n=30). **RQ3-P3 = H0** (P1-perf fails ∧ P2 fails). Honest null: the local agent selector neither beats baselines nor is certifiably non-classifiable.
- AUTHORITATIVE HOLM-7 (frozen family size=7; supersedes lead's conservative partial embedding, D3). RQ2-P3 slot carries the mechanism-corrected primary **H1-pooled Spearman p=0** (not the lead's degenerate p=1). **SURVIVORS: RQ1-P1 (r1×7), RQ2-P1 shrink (r2×6), RQ2-P3 mix (r3×5)** all holm_p=0. NON-survivors: RQ1-P2 (r4×4, holm_p=0.365), RQ3-P2 (r5×3, 0.511), RQ3-P1-perf (r6×2, 0.511), RQ3-P1-latency (r7×1, 0.511). Lead RQ1-P1 & RQ2-P1 survive regardless. Both prereg SHAs intact; $0/offline; worktree-only. NEXT: fill RQ3 half of companion paper (stone 2).
- 2026-07-22 COMPANION PAPER — RQ3 HALF FILLED POST-SEAL (stone 2, D3 combined paper). `docs/stage-07-companion-methods.md`: §5 Results + §6 Discussion RQ3 placeholders AND the authoritative-Holm-7 placeholder REPLACED with real numbers FROM THE SEALED RECORD ONLY (perf 0.6pp CI[1.58,+0.39]pp H0; latency 13.5ms CI[52.1,+34.9] within budget, min-baseline=random; P2 AUC 0.587 CI[0.458,0.703] fingerprint-not-excluded; P3=H0). Authoritative Holm-7 in-paper: survivors RQ1-P1/RQ2-P1/RQ2-P3, non-survivors RQ1-P2/RQ3-P2/RQ3-P1-perf/RQ3-P1-latency. Abstract + top blinding block un-blinded (both tracks). RQ2-P3 half NOT re-litigated (kept as filled at e0d865d). DISCLOSURE carried: 0.93 RQ3 calibration AUC = regime discrimination (churn kp30 vs kp5), DISTINCT from the 0.587 confirmatory selector-vs-selector fingerprint — no HARKing, gates frozen. $0/offline, worktree-only; both prereg SHAs intact. NEXT: report + HOLD for review.
- 2026-07-22 STAGE-08 COMPANION ADVERSARIAL REVIEW (3-persona red-team of combined companion paper). `docs/stage-08-companion-adversarial-review.md` written. METHODOLOGIST: Holm-7 reproduced independently byte-for-byte (survivors RQ1-P1/RQ2-P1/RQ2-P3; monotone step-down lifts ranks 6-7 to 0.511), supersedes-partial + single-slot RQ2-P3 (family_size=7, mechanism-corrected p=0) both sound, run-level bootstrap + two-sided pre-commit valid. DOMAIN SKEPTIC: mix-qualifies-shrink scoped as correction-not-overwrite; calibration preview (ρ→+0.838 dry pass) disclosed + kept DISTINCT from confirmatory +0.6244; RQ3 double-null unspun (perf=ceiling no-headroom, P2=n=30 power limit not all-clear). AUDITOR: EVERY figure matches sealed records (5fdcb379, e09c66ef, 5b61e461); 0.93-vs-0.587 AUC kept distinct (no HARKing); Ollama temp-0 cross-hw caveat present; both prereg SHAs (f22331a72e, 8db4e8a7) recomputed intact; blinding = each track filled ONCE post-seal (git 4f89a90→e0d865d→1b6449b). ONE prose defect fixed (Holm monotonicity clarified, ZERO numbers changed). PASS. $0/offline, worktree-only. NEXT: report + HOLD.