giglez

Trilltechnician/giglez

Fork 0

Commit Graph

Author	SHA1	Message	Date
leetcrypt	f7329df581	feat: benchmark suite + scoring calibration - iteration 4/5 Implements comprehensive benchmarking infrastructure and tunes scoring weights based on empirical accuracy measurements. New Components: - tests/benchmark/test_data_generator.py: Synthetic signal generator for 12 protocols - scripts/benchmark.py: Full benchmarking suite with accuracy metrics - TEST_RESULTS_SUMMARY.md: Detailed benchmark results and per-protocol analysis Benchmark Results: - Total Tests: 12 synthetic signals across 10 protocols - Top-1 Accuracy: 33.3% (4/12 correct) - Top-3 Accuracy: 33.3% - Confidence Distribution: 66.7% high (>80%), 25% medium (50-80%), 8.3% low (<50%) - Avg Processing Time: 144.6ms per signal Scoring Weight Tuning: BEFORE: Timing(30%) + Frequency(25%) + BitCount(20%) + Preamble(15%) + Stats(10%) AFTER: Timing(35%) + Preamble(25%) + BitCount(20%) + Frequency(15%) + Stats(5%) Rationale: - Increased Preamble weight (15% → 25%): Highly discriminative for protocol identification - Increased Timing weight (30% → 35%): Core identification feature - Decreased Frequency weight (25% → 15%): Many protocols share same ISM band - Decreased Stats weight (10% → 5%): Less discriminative in practice Confidence Thresholds: - High: >80% (reliable identification) - Medium: 50-80% (possible match, needs verification) - Low: <50% (uncertain, likely incorrect) Protocol Performance: ✓ Excellent (100%): LaCrosse TX141-BV2, Oregon Scientific v2.1, Schrader TPMS ✗ Needs Improvement (0%): Acurite 609TXC, Princeton, PT2262, Nexus, Toyota TPMS Key Findings: - Preamble detection critical for discrimination (alternating patterns work well) - Timing analysis robust to 15% noise - 315 MHz protocols underrepresented in database - Generic protocols difficult to distinguish without more specific signatures Next Steps (Future Iterations): - Expand 315 MHz protocol coverage - Add protocol-specific heuristics for Princeton, PT2262 - Improve bit pattern matching for similar timing protocols 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>	2026-02-15 07:44:10 -08:00
Trilltechnician	f5d92f1d36	feat: Add wardriving data import system and cleanup tools Major Features: - Multi-format GPS data import (Wigle CSV, GPS JSON, filename GPS) - Data source tracking (test/mock/production) - Database cleanup tools for managing test data - JSON file persistence for simplified server - UI improvements for map controls Backend: - Added wardriving_importer.py with WigleCSVImporter, GPSJSONImporter, BatchImporter - Added data_source and session_id tracking to captures - New API endpoints: DELETE /api/v1/admin/cleanup - Auto-save/load functionality for captures_simple.json - Updated stats endpoint to show data source breakdown CLI Tools: - scripts/import_wardriving_data.py - Batch import with --data-source flag - scripts/cleanup_database.py - Clean by source, session, or all - scripts/create_test_dataset.py - Generate GPS-tagged test data - scripts/test_wardriving_import.py - Test suite for importers Frontend: - Fixed map controls z-index and positioning issues - Moved controls to top-right to avoid zoom button overlap - Fixed Leaflet zoom controls rendering over header - Changed controls to position:fixed for persistent visibility - Export map object to window.map for proper invalidateSize Documentation: - docs/WARDRIVING_IMPORT.md - Complete import guide - docs/DATA_CLEANUP_GUIDE.md - Cleanup system documentation - docs/TEST_LOCATIONS.md - Test GPS coordinates reference - TEST_RESULTS_SUMMARY.md - Format testing results Test Data: - Created test_dataset_gps with 15 captures across 5 US cities - All test data marked with data_source="test" for easy cleanup Testing: - Verified Wigle CSV import (1 capture from West LA) - Verified GPS filename import (3 captures from UCLA area) - Verified GPS JSON companion files (15 captures, 5 cities) - Verified cleanup functionality (deleted 15 test captures) - Verified data persistence across server restarts 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>	2026-01-14 07:48:01 -08:00

Author

SHA1

Message

Date

leetcrypt

f7329df581

feat: benchmark suite + scoring calibration - iteration 4/5

Implements comprehensive benchmarking infrastructure and tunes scoring weights
based on empirical accuracy measurements.

New Components:
- tests/benchmark/test_data_generator.py: Synthetic signal generator for 12 protocols
- scripts/benchmark.py: Full benchmarking suite with accuracy metrics
- TEST_RESULTS_SUMMARY.md: Detailed benchmark results and per-protocol analysis

Benchmark Results:
- Total Tests: 12 synthetic signals across 10 protocols
- Top-1 Accuracy: 33.3% (4/12 correct)
- Top-3 Accuracy: 33.3%
- Confidence Distribution: 66.7% high (>80%), 25% medium (50-80%), 8.3% low (<50%)
- Avg Processing Time: 144.6ms per signal

Scoring Weight Tuning:
BEFORE: Timing(30%) + Frequency(25%) + BitCount(20%) + Preamble(15%) + Stats(10%)
AFTER:  Timing(35%) + Preamble(25%) + BitCount(20%) + Frequency(15%) + Stats(5%)

Rationale:
- Increased Preamble weight (15% → 25%): Highly discriminative for protocol identification
- Increased Timing weight (30% → 35%): Core identification feature
- Decreased Frequency weight (25% → 15%): Many protocols share same ISM band
- Decreased Stats weight (10% → 5%): Less discriminative in practice

Confidence Thresholds:
- High: >80% (reliable identification)
- Medium: 50-80% (possible match, needs verification)
- Low: <50% (uncertain, likely incorrect)

Protocol Performance:
✓ Excellent (100%): LaCrosse TX141-BV2, Oregon Scientific v2.1, Schrader TPMS
✗ Needs Improvement (0%): Acurite 609TXC, Princeton, PT2262, Nexus, Toyota TPMS

Key Findings:
- Preamble detection critical for discrimination (alternating patterns work well)
- Timing analysis robust to 15% noise
- 315 MHz protocols underrepresented in database
- Generic protocols difficult to distinguish without more specific signatures

Next Steps (Future Iterations):
- Expand 315 MHz protocol coverage
- Add protocol-specific heuristics for Princeton, PT2262
- Improve bit pattern matching for similar timing protocols

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2026-02-15 07:44:10 -08:00

Trilltechnician

f5d92f1d36

feat: Add wardriving data import system and cleanup tools

Major Features:
- Multi-format GPS data import (Wigle CSV, GPS JSON, filename GPS)
- Data source tracking (test/mock/production)
- Database cleanup tools for managing test data
- JSON file persistence for simplified server
- UI improvements for map controls

Backend:
- Added wardriving_importer.py with WigleCSVImporter, GPSJSONImporter, BatchImporter
- Added data_source and session_id tracking to captures
- New API endpoints: DELETE /api/v1/admin/cleanup
- Auto-save/load functionality for captures_simple.json
- Updated stats endpoint to show data source breakdown

CLI Tools:
- scripts/import_wardriving_data.py - Batch import with --data-source flag
- scripts/cleanup_database.py - Clean by source, session, or all
- scripts/create_test_dataset.py - Generate GPS-tagged test data
- scripts/test_wardriving_import.py - Test suite for importers

Frontend:
- Fixed map controls z-index and positioning issues
- Moved controls to top-right to avoid zoom button overlap
- Fixed Leaflet zoom controls rendering over header
- Changed controls to position:fixed for persistent visibility
- Export map object to window.map for proper invalidateSize

Documentation:
- docs/WARDRIVING_IMPORT.md - Complete import guide
- docs/DATA_CLEANUP_GUIDE.md - Cleanup system documentation
- docs/TEST_LOCATIONS.md - Test GPS coordinates reference
- TEST_RESULTS_SUMMARY.md - Format testing results

Test Data:
- Created test_dataset_gps with 15 captures across 5 US cities
- All test data marked with data_source="test" for easy cleanup

Testing:
- Verified Wigle CSV import (1 capture from West LA)
- Verified GPS filename import (3 captures from UCLA area)
- Verified GPS JSON companion files (15 captures, 5 cities)
- Verified cleanup functionality (deleted 15 test captures)
- Verified data persistence across server restarts

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

2026-01-14 07:48:01 -08:00

2 Commits