giglez/PLAN_TO_PROD.md

GigLez — Plan to Production

Analyzed: 2026-07-17
Branch: p1-p2-validation
Status: Backend 65% complete · Frontend 5% · Real-world accuracy 0%

---

Critical Findings (Read This First)

The 0% Real-World Problem

The most important finding from benchmark analysis:

Benchmark	Top-1 Accuracy	Source
Synthetic signals	33.3%	scripts/benchmark.py
Real Flipper Zero captures	0.0%	REAL_TEST_RESULTS.md

The matcher produces high-confidence wrong answers on real signals (e.g., LaCrosse TX141 → "Schou 72543 Rain Gauge" at 69% confidence). This is not a threshold issue — it is a fundamental database gap. The RTL_433 protocol set (260+ entries) is dominated by weather sensor timing signatures, so everything pattern-matches to a weather sensor regardless of what it actually is.

Root cause: The rtl433_protocols_imported.py (4540 LOC, the matching backbone) contains only open/known protocols broadcast continuously. Real-world targets like garage door openers, fan controllers, and LED remotes use rolling codes (KeeLoq), proprietary encodings, or undocumented bit structures that have no signatures to match against. These account for 60%+ of Sub-GHz traffic.

Fixing this is the first gating requirement for everything else.

---

Project Snapshot

What's Built (Do Not Rebuild)

Component	File(s)	Status	Quality
.sub / RAW file parser	src/parser/sub_parser.py	✅ Complete	Production-ready
GPS validator	src/gps/validator.py	✅ Complete	22/22 tests pass
ORM models (11 tables)	src/database/models.py	✅ Complete	PostGIS ready
PostgreSQL schema	scripts/create_schema.sql	✅ Complete	Full triggers/indexes
Storage abstraction	src/core/storage/	✅ Complete	FS + S3 backends
Matching engine scaffolding	src/matcher/engine.py	✅ Structure	Needs accuracy fix
5-method scorer	src/matcher/device_identifier.py	✅ Built	Wrong outputs
RTL_433 protocol DB	src/matcher/rtl433_protocols_imported.py	✅ Imported	Incomplete coverage
T-Embed hardware controller	src/capture/tembed.py	✅ Built	Untested live
FastAPI app skeleton	src/api/main_simple.py	🚧 Partial	Routes incomplete
Configuration system	config/settings.py	✅ Complete	Dev/prod aware
ML design doc	docs/ML_DESIGN.md	✅ Designed	Not implemented

What's Missing

Component	Priority	Effort
Accuracy fix (device category DB + protocol routing)	P0	2 days
ML hybrid ensemble (CNN + heuristic)	P1	1 week
Complete API endpoints	P1	3 days
Web UI (map + upload form)	P1	1 week
User auth (JWT + API keys)	P2	2 days
Docker + deployment	P2	1 day
Community features (voting, leaderboard)	P3	1 week
CI/CD pipeline	P3	1 day

---

Device Identification: What Works, What Doesn't

Identifiable Today (with fixes applied)

These device classes have open, documented protocols with stable timing signatures that the current architecture can match:

Device Class	Protocols	Accuracy Potential	Frequency
Weather stations	Oregon Scientific v1/v2/v3, Acurite, LaCrosse	70-85%	433.92 MHz
TPMS (tire pressure)	Schrader, Continental, Pacific, Citroen	60-75%	315/433 MHz
Simple door/window sensors	EV1527, PT2262/2272, Princeton	65-80%	315/433 MHz
Temperature/humidity sensors	Nexus, Ambient Weather, Bresser	70-80%	433 MHz
Rain gauges	Multiple RTL_433-supported	65-75%	433 MHz
Thermostats (simple)	Honeywell, Braeburn	55-70%	345/433 MHz
Smart meters (EU)	Kamstrup, Landis+Gyr	60-70%	868 MHz
LoRa devices (type only)	LoRaWAN chirp signature	50-60%	868/915 MHz

Total identifiable universe: ~350-400 known device types via RTL_433 + Flipper DB

Cannot Be Identified (Protocol-Level Limitation)

Device Class	Why	Workaround
Rolling code garage doors	KeeLoq / Security+ 2.0 — every transmission unique	Can identify device class/family, not specific code
Modern car key fobs	Hopping codes	Frequency + timing only → brand inference
Z-Wave devices	Proprietary 9-byte network ID	Preamble detection → "Z-Wave" category
Zigbee (2.4 GHz)	Out of Sub-GHz range	See 2.4 GHz section below
Fan/LED remotes	Undocumented proprietary	ML category classification helps
Somfy RTS	Proprietary encrypted	Frequency match only (433.42 MHz = Somfy)
DECT cordless phones	1.88-1.9 GHz	Out of range

Identification Approach by Protocol Type

Signal received
    │
    ├─ Has Protocol field (decoded .sub) → Heuristic match → 80-95% accuracy
    │
    ├─ RAW data, known timing signature → RTL_433 protocol match → 60-80%
    │
    ├─ RAW data, unknown timing → ML CNN classifier → 50-70% (device category)
    │
    ├─ Frequency-only match → ISM band category → 30-50% (class only)
    │
    └─ No match → "Unknown Sub-GHz @ {freq}" → stored for community ID

---

Machine Learning Integration

Why ML Is Needed

The heuristic matcher covers open protocols well. ML fills the gap for:

1. RAW signals with no protocol match (fan controllers, LED remotes, proprietary)
2. Device category classification when exact ID is impossible (rolling codes)
3. Manufacturer inference from RF hardware fingerprints
4. Novel protocol discovery — clustering unknown signals

Recommended Architecture (from docs/ML_DESIGN.md)

Implement the designed Hybrid Ensemble in phases:

Phase A: Statistical Features (Lowest effort, highest ROI)

Train a Random Forest / Gradient Boosting classifier on 47 extracted features:

• Pulse width statistics (mean, std, min, max SHORT/LONG)
• LONG/SHORT ratio
• Pulse count, repetition count
• Preamble pattern type
• ISM band classification
• Duty cycle, inter-burst gap

# Feature vector already partially built in:
# src/matcher/statistical_classifier.py (356 LOC)
# src/matcher/timing_analyzer.py (550 LOC)

# Target: device category (not model)
# Classes: weather_sensor, garage_door, key_fob, doorbell, 
#          security_sensor, fan_controller, tpms, smart_meter, unknown

Training data: UberGuidoZ Flipper DB (10,000+ labeled .sub files) — directory structure provides labels.
Expected accuracy: 60-75% category-level
Effort: 3-4 days (feature extraction exists, need training pipeline)

Phase B: 1D CNN on Raw Pulse Arrays

Input: normalized pulse array → padded to 512 samples
Architecture:
  Conv1D(64, 3) → BatchNorm → ReLU
  Conv1D(128, 3) → BatchNorm → ReLU  
  Conv1D(256, 3) → BatchNorm → ReLU
  GlobalAvgPool
  Dense(256) → Dropout(0.3)
  Dense(N_classes) → Softmax

Output: P(device_category) for 9 categories

Training: PyTorch or TensorFlow (Python training, ONNX export for serving)
Inference: Python via ONNX Runtime (server-side, NOT browser — model too complex for TF.js)
Expected accuracy: 70-80% category-level
Effort: 5-7 days

Phase C: Ensemble Combiner

final_score = (
    0.40 * heuristic_score +   # existing matcher (exact protocols)
    0.35 * cnn_score +          # CNN category probability  
    0.25 * stat_score           # statistical features
)

The heuristic matcher dominates for known protocols; ML fills gaps for unknowns.

What ML Cannot Do

• Decode rolling code content (cryptographically protected)
• Identify specific device model from RAW alone (insufficient features)
• Work on BinRAW without normalization
• Achieve >90% model-level accuracy with Sub-GHz RAW (physics limitation)

Realistic production targets:

• Known protocol devices: 80-90% top-1 accuracy (heuristic)
• Unknown/proprietary: 60-75% device category (ML)
• Rolling code: 40-55% manufacturer/family inference

---

2.4 GHz / 5 GHz / 6 GHz Feasibility

Technical Reality

Band	Hardware Needed	Protocol	File Format	Flipper Support
Sub-GHz (300-928 MHz)	Flipper Zero, RTL-SDR	OOK/FSK/ASK	.sub	✅ Native
2.4 GHz	WiFi card (monitor mode)	802.11 b/g/n, Zigbee, BT	.pcap / .pcapng	❌ Hardware limit
5 GHz	WiFi card (monitor mode)	802.11 a/n/ac	.pcap	❌ Hardware limit
6 GHz	WiFi 6E card	802.11ax	.pcap	❌ Hardware limit

Can GigLez Support 2.4/5/6 GHz?

Yes, but as a separate ingestion path. The platform architecture (GPS + file + database) generalizes cleanly. What changes:

1. Input format: Accept .pcap/.pcapng instead of .sub
2. Parser: pyshark or scapy to extract beacon/probe frames (already in ~/coding/_archive/pyshark)
3. Identification: Network fingerprinting by:
   • SSID patterns → device type (e.g., DECO-XXXX → TP-Link mesh)
   • OUI lookup from MAC → manufacturer
   • IE (Information Elements) → chipset inference
4. Database: Separate wifi_captures table alongside captures

Wigle does exactly this for WiFi — their schema can be adapted directly.

Recommendation: Phase 4 feature. Sub-GHz is the differentiator; WiFi is Wigle's territory. Add 2.4/5/6 GHz support after MVP to expand data collection surface, not before.

Effort: 1 week to add WiFi pcap ingestion + OUI database + map layer.

---

Production Roadmap

Phase 0: Fix Accuracy (Gating — 1 week)

This blocks everything. No point shipping a product that gives wrong answers with high confidence.

0.1 — Build device category routing layer (2 days)

Create src/matcher/category_router.py that classifies before trying to match:

# Frequency + timing → category → route to specialized sub-matcher
# 315 MHz + short TE < 500 → key_fob_matcher
# 433 MHz + long burst preamble → weather_matcher  
# 433 MHz + EV1527 timing → remote_matcher

0.2 — Expand signature database (2 days)

The current DB has RTL_433 protocols. Add:

• Princeton, EV1527, PT2262 parameter ranges (simple remotes — largest category)
• KeeLoq family detection (rolling code → don't match model, classify family)
• Flipper Zero firmware .sub collection bulk import (UberGuidoZ: 10K+ files)
  • Parse filenames as labels: Garage_Doors/LiftMaster/*.sub → category + brand
• Z-Wave preamble signature (868/908 MHz chirp pattern)

0.3 — Add confidence calibration (1 day)

Current system produces 69-76% confidence for wrong answers. Add isotonic regression calibration or simply apply hard rules:

• If top-1 and top-2 confidence differ by < 5% → reduce both to "low"
• If frequency doesn't match device category → penalize 30%
• If timing LONG/SHORT ratio is outside protocol's known range → cap at 50%

0.4 — Re-run real-world benchmark (0.5 days)

Target: Top-3 accuracy ≥ 30% on REAL_TEST_RESULTS.md test set. This is achievable with routing alone.

---

Phase 1: Complete the Backend API (1 week)

All routes exist as skeletons. Wire them up.

1.1 — POST /api/v1/captures/upload (2 days)

• Accept multipart: JSON manifest + .sub files
• Validate GPS (already done — src/gps/validator.py)
• Parse files (already done — src/parser/sub_parser.py)
• Run matcher (fixed in Phase 0)
• Store in PostgreSQL
• Return capture IDs + match results

1.2 — GET /api/v1/search (1 day)

• Query by bounding box (PostGIS ST_Within)
• Filter by: frequency range, protocol, device category, date range
• Pagination (cursor-based, Wigle pattern)
• Return GeoJSON for map rendering

1.3 — GET /api/v1/heatmap (0.5 days)

• Aggregate by grid cell (H3 hexagons or lat/lon bins)
• Use materialized view capture_heatmap (already designed in schema)
• Return density + dominant device category per cell

1.4 — GET /api/v1/stats (0.5 days)

• Total captures, unique devices, geographic coverage
• Top frequencies, protocols, device categories
• Uses materialized view device_statistics

1.5 — GET /api/v1/devices (1 day)

• List/search known device catalog
• Filter by manufacturer, category, frequency
• Device detail page (signatures, captures seen)

1.6 — Auth middleware (1 day)

• Anonymous uploads allowed (rate-limited by IP)
• Optional JWT for account-linked uploads
• API key header for programmatic access
• FastAPI dependency injection (clean pattern)

---

Phase 2: Web Interface MVP (1 week)

Minimum: something users can open in a browser and actually use.

2.1 — Upload form (2 days)

• Drag-and-drop .sub files
• GPS entry: manual lat/lon OR "use browser location"
• Timestamp auto-fill
• Upload progress + results display
• Simple HTML/CSS + vanilla JS (no framework needed for MVP)

2.2 — Interactive map (3 days)

• Leaflet.js (already in plan, free, no API key)
• Marker clustering (Leaflet.markercluster)
• Click → device details panel
• Heatmap toggle (Leaflet.heat)
• Filter sidebar: frequency, category, date

2.3 — Basic pages (2 days)

• / — Landing + recent captures
• /map — Interactive map
• /upload — Upload form
• /devices — Device catalog
• /stats — Platform statistics

Use Jinja2 templates (FastAPI native) + Tailwind CSS via CDN. No build step needed.

---

Phase 3: ML Integration (1 week, parallel-able with Phase 2)

Implement Phase A + B from the ML strategy above.

3.1 — Training data pipeline (2 days)

# Download UberGuidoZ Flipper database (public, ~2GB)
# Parse directory structure as labels
# Extract 47 statistical features per file
# Split 80/10/10 train/val/test

3.2 — Train statistical classifier (1 day)

# sklearn GradientBoostingClassifier or XGBoost
# 9 device categories
# Save as joblib model
# Expected: 60-75% category accuracy

3.3 — Train 1D CNN (2 days)

# PyTorch, pad RAW arrays to 512 samples  
# Train 20 epochs on GPU (if available) or CPU (~2h)
# Export to ONNX
# Expected: 70-80% category accuracy

3.4 — Wire ensemble into matcher (1 day)

• Load ONNX model at startup
• Run in parallel with heuristic matcher (async)
• Combine scores with 40/35/25 weighting
• Store ML category alongside heuristic matches in capture_matches table

3.5 — Active learning hook (0.5 days)

• Community verifications → retrain queue
• Simple: flag captures with low confidence + community correction
• Retrain monthly

---

Phase 4: Dockerize + Deploy (3 days)

4.1 — Docker Compose (1 day)

services:
  api:        # FastAPI + uvicorn
  db:         # PostgreSQL 16 + PostGIS
  redis:      # Cache + rate limiting
  worker:     # Celery for async file processing
  nginx:      # Reverse proxy + static files

4.2 — Deployment target (1 day)

• VPS: 4 vCPU, 8GB RAM, 100GB SSD (enough for 10M captures)
• PostgreSQL tuned for geospatial workloads
• Nginx + Let's Encrypt SSL
• systemd service for auto-restart

4.3 — Operational tooling (1 day)

• Health check endpoint
• Prometheus metrics (/metrics)
• Log rotation (loguru already wired)
• DB backup cron (daily dump to S3)

---

Phase 5: Community Features (2 weeks, post-launch)

5.1 — User accounts (3 days)

• Registration/login (email or anonymous)
• Upload history + statistics
• API key management
• Reputation score

5.2 — Device verification (3 days)

• "Confirm" / "Dispute" buttons on matches
• Voting system (votes table already in schema)
• High-vote matches → promoted to "verified"
• Community corrections feed ML retraining

5.3 — Leaderboards + badges (2 days)

• Top uploaders (by count, by new devices discovered)
• Geographic coverage leader
• Milestone badges (first in city, 100 captures, etc.)

5.4 — 2.4/5 GHz expansion (5 days)

• Accept .pcap/.pcapng uploads
• pyshark-based parser for WiFi beacons
• OUI database lookup (manufacturer from MAC)
• New map layer: WiFi devices
• Wigle-compatible CSV export

---

Tech Stack Gap Analysis

Has / Doesn't Need Replacing

Component	Current	Status
Parser	Custom Python	✅ Keep
GPS validation	Custom Python	✅ Keep
Database	PostgreSQL + PostGIS	✅ Keep (needs init)
ORM	SQLAlchemy 2.0	✅ Keep
API framework	FastAPI 0.109	✅ Keep
Storage	FS (dev) / S3 (prod)	✅ Keep
Config	python-dotenv	✅ Keep

Needs Adding

Component	Recommendation	Why
Map library	Leaflet.js 1.9	Free, no API key, proven
ML training	PyTorch + scikit-learn	Already in requirements
ML inference	ONNX Runtime	Lightweight, no GPU required
Caching	Redis (via redis-py)	Rate limiting + query cache
Task queue	Celery + Redis	Async file processing
Frontend CSS	Tailwind CSS (CDN)	No build step for MVP
Container	Docker + Compose	Reproducible deployment
Reverse proxy	Nginx	SSL + static files

Remove / Simplify

Item	Action
src/api/main_orm_legacy.py	Delete — superseded
src/api/routes/captures_enhanced.py	Merge into captures.py, delete
src/api/routes/captures_with_logging.py	Merge into captures.py, delete
src/matcher/strategies.py (610 LOC legacy)	Keep as fallback but don't extend
scripts/archive/	Leave archived — don't delete

---

Honest Timeline to Production

Assuming 4-6 focused hours/day:

Phase	What	Duration	Cumulative
0	Fix accuracy (critical)	1 week	Week 1
1	Complete backend API	1 week	Week 2
2	Web UI MVP	1 week	Week 3
3	ML integration	1 week	Week 3-4 (parallel)
4	Docker + deploy	3 days	Week 4
MVP Live	Public beta		~Week 4
5	Community features	2 weeks	Week 6
WiFi	2.4/5 GHz expansion	1 week	Week 7

Minimum viable launch (something to show GhostArmyIntel): 3-4 weeks

---

Key External Resources for Phase 0

Resource	URL	Use
UberGuidoZ Flipper DB	github.com/UberGuidoZ/Flipper	10K+ labeled .sub training files
RTL_433 test files	github.com/merbanan/rtl_433/tree/master/tests	Labeled real captures
RadioML dataset	deepsig.ai/datasets	ML training for modulation recognition
Protocol table	github.com/merbanan/rtl_433/blob/master/README.md	260+ protocol reference
Flipper firmware protocol list	github.com/flipperdevices/flipperzero-firmware/tree/dev/lib/subghz/protocols	Canonical protocol params

---

Success Criteria (Per Phase)

Phase	Gate
Phase 0 complete	Real-world top-3 accuracy ≥ 30% (up from 0%)
Phase 1 complete	POST /upload → stored in DB → returns matches
Phase 2 complete	User can upload .sub file in browser, see it on map
Phase 3 complete	ML model reduces "unknown" rate from 45% → < 25%
Phase 4 complete	docker compose up → production site live on VPS
Phase 5 complete	100+ community-verified identifications

---

Generated from deep analysis of project state + docs/ML_DESIGN.md + REAL_TEST_RESULTS.md + docs/RF_SIGNAL_ANALYSIS_RESEARCH.md + docs/FREQUENCY_DEVICE_CHART.md on 2026-07-17.