feat: Phase 3A statistical device-category classifier (86.8% top-1 on RAW)

Browse Source

Trains sklearn RandomForest + GradientBoosting on 14 timing/statistical
features from RAW-only UberGuidoZ captures, with a group-aware split
(GroupShuffleSplit by device sub-folder) so near-duplicate captures never
leak across train/test. The heuristic CategoryRouter is scored on the exact
same held-out files for a fair comparison.

Best model (gradient_boost): 86.8% top-1 accuracy, 62.5% balanced accuracy
vs heuristic 30.0% top-1 / 55.9% routed. Raw accuracy is inflated by the
70%-Garage RAW class balance; balanced accuracy (~2x the heuristic) is the
honest number and meets the Phase 3A 60-75% target. Not yet wired into the
decoder ensemble.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

...

This commit is contained in:

leetcrypt

2026-07-19 11:59:13 -07:00

parent 9f32e448ab

commit 67c376e92c

3 changed files with 307 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

models/category_classifier.joblib

BIN

Binary file not shown.