Adds a rules-based device category router that classifies a signal by
frequency band + timing ratio + pulse count BEFORE the per-protocol
scoring loop, restricting the candidate set. This fixes the "everything
matches a weather sensor with 69-76% false confidence" problem.
- src/matcher/category_router.py: frequency-band + timing routing
- pattern_decoder.py: category filter, category-mismatch penalty,
post-match spread penalty (surfaces low-discrimination cases)
- protocol_database.py: garage door / doorbell / fan controller entries
- scripts/benchmark_phase0.py: real-world-shaped benchmark
Benchmark gate: top-3 accuracy 0% -> 67% (target >=30%). 52/52 unit
tests pass. NOTE: benchmark .sub files are synthetic-from-DB-params,
so 67% is an upper bound pending real Flipper capture validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## Phase 1: Code Cleanup (~1,859 lines removed)
**Deleted Redundant Matchers:**
- ❌ strategies_orm.py (356 lines) - Old ORM-based strategies
- ❌ simple_matcher.py (351 lines) - Replaced by strategies.py
- ❌ rtl433_matcher.py (352 lines) - Replaced by strategies.py
**Archived Old Scripts:**
- Moved 11 one-time analysis/import scripts to scripts/archive/
- Scripts: analyze_flipper_signatures, analyze_tembed_files, identify_tembed_devices,
import_flipper_sqlite, import_tembed_signatures, match_tembed_with_db,
match_with_flipper_db, rematch_captures, test_gps_extraction,
test_tembed_matching, test_wardriving_import
**Consolidated API:**
- Renamed main.py → main_orm_legacy.py (archived old ORM-based API)
- main_simple.py is now the primary production API
## Phase 2: Pattern Decoder Integration ✅
**CRITICAL FIX: Pattern decoder now integrated into production API!**
**Changes:**
1. Updated main_simple.py to use unified SignatureMatcher
2. Added 6 strategies to matcher pipeline:
- ExactMatcher (protocol + frequency)
- FrequencyMatcher (frequency-based)
- BitPatternMatcher (data patterns)
- TimingMatcher (timing-based)
- RTL433DecoderStrategy (RTL_433 decoder)
- PatternBasedStrategy (NEW! Pattern decoder for short captures)
3. Created MockDB class for simplified mode (no real database)
4. Replaced old get_matcher() with get_matcher_engine()
5. Updated upload handler to use MatchResult format
6. All matches now include confidence scores and match methods
**Result:**
- Pattern decoder is NOW ACTIVE in production 🎉
- Unified matching pipeline with 6 strategies
- Cleaner codebase (-1,859 lines)
- Single source of truth for matching logic
🎉 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Created RF_TEST_DATABASES.md with 6 primary databases (30,000+ files)
- Added RTL433_TESTING_GUIDE.md explaining 0% decode rate
- Created download_rf_test_datasets.sh to fetch all test data
- Documented web search results from Google dorking
- Listed repositories: rtl_433_tests, FlipperZero-Subghz-DB, UberGuidoZ, etc
- Included testing strategy and expected success rates
Major Features:
- Multi-format GPS data import (Wigle CSV, GPS JSON, filename GPS)
- Data source tracking (test/mock/production)
- Database cleanup tools for managing test data
- JSON file persistence for simplified server
- UI improvements for map controls
Backend:
- Added wardriving_importer.py with WigleCSVImporter, GPSJSONImporter, BatchImporter
- Added data_source and session_id tracking to captures
- New API endpoints: DELETE /api/v1/admin/cleanup
- Auto-save/load functionality for captures_simple.json
- Updated stats endpoint to show data source breakdown
CLI Tools:
- scripts/import_wardriving_data.py - Batch import with --data-source flag
- scripts/cleanup_database.py - Clean by source, session, or all
- scripts/create_test_dataset.py - Generate GPS-tagged test data
- scripts/test_wardriving_import.py - Test suite for importers
Frontend:
- Fixed map controls z-index and positioning issues
- Moved controls to top-right to avoid zoom button overlap
- Fixed Leaflet zoom controls rendering over header
- Changed controls to position:fixed for persistent visibility
- Export map object to window.map for proper invalidateSize
Documentation:
- docs/WARDRIVING_IMPORT.md - Complete import guide
- docs/DATA_CLEANUP_GUIDE.md - Cleanup system documentation
- docs/TEST_LOCATIONS.md - Test GPS coordinates reference
- TEST_RESULTS_SUMMARY.md - Format testing results
Test Data:
- Created test_dataset_gps with 15 captures across 5 US cities
- All test data marked with data_source="test" for easy cleanup
Testing:
- Verified Wigle CSV import (1 capture from West LA)
- Verified GPS filename import (3 captures from UCLA area)
- Verified GPS JSON companion files (15 captures, 5 cities)
- Verified cleanup functionality (deleted 15 test captures)
- Verified data persistence across server restarts
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
Major milestone: GPS coordinates now auto-extract from filenames and
uploads appear on map with full end-to-end workflow functional!
✨ GPS Auto-Extraction Features:
- JavaScript GPS extractor class matching Python patterns
- Supports 3 filename formats:
* N/S/E/W: 34.0478N_118.2349W_filename.sub
* lat/lon prefix: lat34.0478lon-118.2348_filename.sub
* Signed decimal: -34.0478_118.2348_filename.sub
- Auto-populates latitude/longitude form fields on file drop
- Green notification toast shows detected coordinates
- File list shows GPS badge for files with coordinates
🗺️ Web Interface Improvements:
- Upload endpoint now stores captures in-memory
- Query endpoint returns uploaded captures for map display
- Stats endpoint shows real-time upload counts
- Map displays uploaded captures as markers
- Color-coded by frequency band
📁 Updated Files:
- static/js/upload.js: GPS extraction + auto-population
- src/api/main_simple.py: In-memory storage + endpoints
- src/parser/gps_extractor.py: Backend GPS extraction (Python)
- scripts/test_gps_extraction.py: Python test suite
- test_gps_extraction.html: Browser test suite
📊 T-Embed Files Updated:
- 34.0478N_118.2348W_1637_raw_8.sub: 315 MHz Princeton
- 34.0478N_118.2349W_1351_raw_10.sub: 433.92 MHz Princeton
- 34.0478N_118.2349W_1650_test_raw.sub: 433.92 MHz RAW
- All now have proper Flipper SubGhz headers
✅ Tested Features:
- GPS extraction from filename: 34.0478N_118.2349W → 34.0478, -118.2349
- Auto-population of GPS fields in upload form
- File upload with GPS validation
- Capture appears on map after upload
- Statistics update in real-time
- Frequency distribution calculated correctly
🎯 End-to-End Flow Working:
1. User drops .sub file with GPS in filename
2. GPS auto-detected and form fields populate
3. User clicks Upload
4. Server parses RF data + GPS coordinates
5. Capture stored in memory
6. Map refreshes and displays new marker
7. Stats update with new counts
🚀 Demo: http://localhost:8000
Upload 34.0478N_118.2349W_1351_raw_10.sub and watch it appear on map!
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
leetcrypt
Trilltechnician