leetcrypt 67c376e92c feat: Phase 3A statistical device-category classifier (86.8% top-1 on RAW) ...

Trains sklearn RandomForest + GradientBoosting on 14 timing/statistical
features from RAW-only UberGuidoZ captures, with a group-aware split
(GroupShuffleSplit by device sub-folder) so near-duplicate captures never
leak across train/test. The heuristic CategoryRouter is scored on the exact
same held-out files for a fair comparison.

Best model (gradient_boost): 86.8% top-1 accuracy, 62.5% balanced accuracy
vs heuristic 30.0% top-1 / 55.9% routed. Raw accuracy is inflated by the
70%-Garage RAW class balance; balanced accuracy (~2x the heuristic) is the
honest number and meets the Phase 3A 60-75% target. Not yet wired into the
decoder ensemble.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-07-19 11:59:13 -07:00

732 KiB

Raw Permalink History

View Raw