Trilltechnician/giglez
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
24 Commits 1 Branch 0 Tags
9be14af16012e58feed7ed37295714ec0657b86f
Commit Graph

6 Commits

Author SHA1 Message Date
Trilltechnician 9be14af160 Implement pattern-based decoder for single-transmission RF captures 
Added pattern-based decoding system specifically designed for short captures
from Flipper Zero and LilyGo T-Embed devices that don't have enough repetitions
for RTL_433.

## New Components:

1. **Protocol Database** (protocol_database.py)
   - 18 known RF protocol signatures
   - Categories: Weather Sensors, Garage Doors, TPMS, Doorbells, etc.
   - Timing patterns for Acurite, Oregon Scientific, LaCrosse, Nexus, etc.

2. **Pattern Decoder** (pattern_decoder.py)
   - Multi-strategy decoder using 3 approaches:
     - Timing pattern analysis (K-means clustering for SHORT/LONG pulses)
     - Statistical fingerprinting (signal characteristics)
     - Protocol library matching
   - Works with single-transmission captures (100-500 pulses)

3. **Matcher Integration** (strategies.py)
   - Added PatternBasedStrategy to matcher pipeline
   - Integrates with existing MatchResult system
   - Confidence scoring: 0.5-0.9 based on match quality

## Test Results:

**Pattern Decoder vs RTL_433 Performance:**
- RTL_433: 0% decode rate (0/8 files) - requires multiple repetitions
- Pattern Decoder: 44.4% decode rate (4/9 files) - works with single captures

**Successful Decodes:**
- Oregon Scientific weather sensors (76% confidence)
- Acurite weather stations (52% confidence)
- 24 total device matches across 4 files

## Implementation Details:

- K-means clustering for pulse width identification
- Statistical fingerprinting with mean, std, duty cycle
- Protocol database with 7 weather sensors + 11 other device types
- Confidence thresholds optimized for single-tx captures
- Fallback to sklearn K-means or percentile-based clustering

## Documentation:

- PATTERN_BASED_DECODING_PLAN.md: Complete implementation plan
- test_pattern_decoder.py: Comprehensive test suite

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-01-14 19:22:07 -08:00
Trilltechnician 8560fb5002 feat: Complete RTL_433 integration (Phases 1-8) 
Implemented full RTL_433 decoder integration for automatic IoT device identification:

## Phase 1-6: Core Implementation
- RTL_433 binary integration (v23.11, 244 protocols)
- Pulse data converter (RAW_Data → am.s16 format)
- Subprocess decoder wrapper with JSON parsing
- RTL433DecoderStrategy for matcher pipeline
- Comprehensive test suite (all tests passing)

## Phase 8: API Integration (this commit)
- GET /api/rtl433/status - Check decoder availability
- GET /api/rtl433/protocols - List 244 supported protocols
- GET /rtl433/protocols/{id} - Get protocol info
- POST /api/v1/captures/upload - Updated with RTL_433 decoding

## Files Added
- src/parser/rtl433_converter.py (~300 lines)
- src/matcher/rtl433_decoder.py (~400 lines)
- src/matcher/strategies.py (RTL433DecoderStrategy)
- docs/RTL433_INTEGRATION_PLAN.md
- docs/RTL433_IMPLEMENTATION_STATUS.md
- docs/RTL433_API_ENDPOINTS.md
- docs/PHASE8_COMPLETE.md
- tests/test_rtl433_integration.py
- tests/test_rtl433_api.sh

## Files Modified
- src/api/main_simple.py - Added RTL_433 decoding to upload
- src/api/routes/hardware.py - Added RTL_433 endpoints

## Performance
- Decode time: <0.05s per file
- 244 protocols supported
- 0.95 confidence for successful decodes

## Testing
- All integration tests passing
- All API endpoint tests passing
- ~3,600+ lines of code & documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-01-14 17:59:43 -08:00
Trilltechnician de9dcda1f7 feat: Phase 2 & 3 - RTL_433 integration + RAW timing analysis 
Phase 2: RTL_433 Protocol Matcher (286 devices)
================================================
Created src/matcher/rtl433_matcher.py
- RTL433Matcher class with JSON database loader
- Built search indexes: device_id, name, category, modulation
- Fuzzy matching with difflib.SequenceMatcher (>0.6 similarity)
- Timing signature matching (±15% tolerance)
- Confidence scoring:
  * Exact ID match: 0.95
  * Exact name match: 0.90
  * Fuzzy match: 0.70-0.85
  * Timing match: 0.70-0.95
- Singleton pattern for performance

Phase 3: RAW Signal Timing Analysis
====================================
Created src/parser/raw_parser.py
- RAWParser class for Flipper Zero RAW_Data format
- TimingSignature dataclass with pulse analysis
- Extracts short_pulse, long_pulse, gap, pulse_ratio
- Percentile-based clustering (25th/75th)
- Encoding detection (PWM, PPM, Manchester, OOK)
- Statistical analysis (mean, std, total duration)

Integration & Enhancements
===========================
Enhanced src/matcher/simple_matcher.py
- Added raw_data parameter to match() method
- RTL_433 protocol matching (Phase 2) with logging
- Timing analysis for RAW captures (Phase 3)
- Graceful degradation with try/except
- RTL433_AVAILABLE flag for feature detection
- Maintains backward compatibility

Updated src/api/main_simple.py
- Extract raw_data from parsed metadata
- Convert List[int] to space-separated string
- Pass raw_data to enhanced matcher

Validation Results
==================
Test script: test_enhanced_matcher.py
- 20 existing captures re-matched
- 4 captures improved (20%)
- 0 captures worse (0%)
- Average improvement: +0.16 confidence
- Best improvement: +0.35 (MegaCode → Linear Megacode)
- RTL_433 exact match: MegaCode → 0.95 confidence
- RTL_433 fuzzy match: Princeton → Insteon 0.79

Expected Accuracy
=================
- Phase 2 alone: 75-80% (+15%)
- Phase 2 + 3: 80-85% (+20-25%)
- Current validation: Phase 2 confirmed working
- Phase 3: Requires new uploads with raw_data

Deployment Ready
================
- Backward compatible (optional raw_data)
- No breaking changes to API
- Graceful import fallback
- Ready for server deployment
 2026-01-14 12:01:42 -08:00
Trilltechnician b083890e96 feat: Integrate frequency-based device identification system 
Integrated comprehensive device attribution system into GigLez:

1. Simple Device Matcher (src/matcher/simple_matcher.py):
   - Frequency-based device categorization (315/433/868/915 MHz)
   - Protocol-specific identification (Princeton, EV1527, Oregon Scientific, etc.)
   - Modulation + frequency matching (OOK/FSK/ASK)
   - Confidence scoring (0.4-0.95 range)
   - 50+ device types covered

2. API Integration (src/api/main_simple.py):
   - Device matching in upload pipeline
   - Added device fields: device_name, device_category, match_confidence, match_method, device_description
   - Top 5 alternative matches stored per capture
   - New endpoint: GET /api/v1/captures/{id} for detail view

3. Frontend Implementation:
   - Detail modal with comprehensive device information
   - Device identification section with confidence bars
   - Alternative matches display
   - Signal, location, and metadata sections
   - Keyboard (ESC) and click-outside modal closing

4. UI Enhancements (static/css/main.css):
   - Modal overlay with backdrop blur
   - Animated modal slide-in
   - Confidence visualization (green/yellow/red bars)
   - Responsive detail grid layout
   - Device match cards with categories

5. JavaScript Integration:
   - detail-modal.js: Comprehensive detail view renderer
   - Updated map.js and search.js to use detail modal
   - Removed placeholder functions

6. Utilities:
   - scripts/rematch_captures.py: Re-run matcher on existing data
   - Successfully re-matched 20 existing captures

Device Categories Supported:
- Consumer RF (remotes, sensors)
- Automotive (key fobs, TPMS)
- Home Automation (garage/gate openers, blinds)
- Sensors (weather stations, temperature)
- Security (door/window sensors, alarms)
- IoT (smart meters, LoRa devices)
- Industrial (SCADA, telemetry, RFID)

Match Methods:
- Protocol matching (highest confidence: 0.7-0.95)
- Frequency matching (0.4-0.7)
- Modulation + frequency matching (0.5-0.7)

Frontend now displays:
- Device name and category on map markers
- Confidence percentage
- Detailed device information modal
- Alternative device matches
- Match method explanation

All existing captures successfully identified with 60-70% confidence.
 2026-01-14 10:51:46 -08:00
Trilltechnician 48fcb00241 Phase 3 Complete: Web Interface MVP 
Major Achievements:
- ✅ Full web interface (1,520+ lines of frontend code)
- ✅ Interactive Leaflet.js map with marker clustering
- ✅ Drag-and-drop upload system with GPS input
- ✅ Search & filter UI with multi-criteria
- ✅ Statistics dashboard with Chart.js
- ✅ Responsive mobile-friendly design

Backend:
- ✅ FastAPI static file serving
- ✅ Simplified server mode (main_simple.py)
- ✅ Improved startup script with port auto-selection
- ✅ PostgreSQL schema ready (requires setup)

Database:
- ✅ SQLite populated with 85 Flipper Zero signatures
- ✅ Device matching system operational
- ✅ Frequency-based search working

Documentation:
- ✅ PHASE_3_COMPLETE.md - Technical summary
- ✅ WEB_INTERFACE_README.md - User guide
- ✅ WEBAPP_STARTUP_GUIDE.md - Troubleshooting
- ✅ POSTGRESQL_SETUP_EXPLANATION.md - DB setup guide
- ✅ DATABASE_POPULATION_SUCCESS.md - Import report
- ✅ DEVICE_IDENTIFICATION_REPORT.md - Matching analysis

Files Created:
- templates/index.html (260 lines)
- static/css/main.css (500 lines)
- static/js/*.js (760 lines total)
- src/api/main_simple.py (simplified server)
- start_web.sh (auto port selection)

Status: Production MVP Ready
Next: Phase 4 - API & Integration

🛰️ Generated with Claude Code
https://claude.com/claude-code

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-01-12 18:21:11 -08:00
Trilltechnician eb225771bc Initial commit: Phase 1 & Phase 2 infrastructure complete 2026-01-12 11:21:17 -08:00