giglez/docs/REFOCUS_SUMMARY.md

Project Refocus Summary

Date: 2025-01-11 Major Pivot: Hardware-specific → Platform-agnostic approach

What Changed

Before: Device-Dependent Architecture

• Focused on LilyGo T-Embed integration
• Termux + Android GPS as primary setup
• Serial communication protocols
• Hardware-specific capture workflow

After: Platform-Agnostic Web Service

• Accept uploads from ANY capture device
• .sub/.fff file submission with GPS coordinates
• Web platform like Wigle.net
• Focus on parsing, matching, and visualization

Key Principle

We don't care what hardware you use. If you can generate .sub files with GPS coordinates, you can contribute to GigLez.

Core Platform Features

1. File Upload System

• Input: .sub/.fff files + GPS coordinates
• Submission Methods:
  • Web drag-and-drop interface
  • REST API (POST /api/submit)
  • Batch uploads (ZIP + manifest.json)
• Anonymous or Authenticated: User's choice

2. Automatic Device Identification

.sub File Parser (src/parser/)

Extracts metadata from Flipper Zero .sub files:

• Frequency, protocol, modulation
• Bit length, key data, timing
• RAW signal data (timing arrays)
• Custom preset configurations

Files Created:

• src/parser/metadata.py - Data structures for signal metadata
• src/parser/sub_parser.py - Complete .sub file parser with validation

Signature Matching Engine (src/matcher/)

Multi-strategy matching against known devices:

1. ExactMatcher (100% confidence)

   • Protocol + Frequency + Bit Length

2. PartialMatcher (80% confidence)

   • Protocol + Frequency only

3. PatternMatcher (70-90% confidence)

   • Bit pattern similarity with masks
   • Hamming distance calculations

4. TimingMatcher (60-80% confidence)

   • Pulse timing characteristics from RAW files
   • Average pulse width matching

5. FrequencyMatcher (50-70% confidence)

   • Fuzzy frequency matching within tolerance

Files Created:

• src/matcher/engine.py - Main matching coordinator
• src/matcher/strategies.py - All matching strategy implementations

3. Wigle.net-Inspired Platform

Analyzed Features

Wigle.net Feature	GigLez Implementation
CSV upload format	.sub file upload + GPS JSON/CSV
349M+ WiFi networks	Start with IoT RF devices
Text search (SSID/MAC)	Search by device type/protocol
Geographic bounding box	PostGIS geospatial queries
Heatmap visualization	Device density by location
User leaderboards	Contribution tracking + reputation
API access	RESTful JSON API

Key Differences

• Wigle: Exact identification (MAC address = unique)

• GigLez: Fuzzy matching (signal patterns → likely device)

• Wigle: Public by default

• GigLez: Opt-in sharing, anonymization options

Documentation Updates

CLAUDE.md (Primary Directive)

Complete rewrite focusing on:

• Platform-agnostic submission workflow
• Wigle.net analysis and implementation strategy
• .sub file parsing and matching pipeline
• System architecture diagram
• Submission API format specs

Removed: All T-Embed/Termux/hardware-specific references

README.md

Now emphasizes:

• "Platform-agnostic web service"
• Supported capture devices (Flipper, HackRF, RTL-SDR, etc.)
• Quick start with web upload or API
• Submission format examples
• Wigle.net comparison table

New sections:

• How It Works (diagram)
• Supported Capture Devices
• API Documentation
• Privacy & Security

Technical Implementation

Completed Components

1. .sub File Parser

from src.parser import parse_sub_file

metadata = parse_sub_file('capture.sub')
# Returns: SignalMetadata with frequency, protocol, modulation, etc.

Features:

• Handles KEY, RAW, and BinRAW formats
• Extracts timing patterns from RAW files
• Validates file structure
• Error collection and reporting

2. Signature Matching Engine

from src.matcher import SignatureMatcher

matcher = SignatureMatcher(database)
matcher.add_strategy(ExactMatcher())
matcher.add_strategy(PartialMatcher())
matcher.add_strategy(PatternMatcher())

matches = matcher.match(metadata, max_results=10)
# Returns: List[MatchResult] with confidence scores

Features:

• Pluggable strategy pattern
• Automatic deduplication
• Confidence-based sorting
• Detailed match explanations

Database Schema (Already Complete)

• captures - RF captures with GPS
• devices - Known device catalog
• signatures - Matching patterns (Flipper/RTL_433)
• identifications - Community verifications
• PostGIS for geospatial queries

Signature Databases (Documented)

• Flipper Zero: 1000+ .sub files
• RTL_433: 200+ protocol definitions
• Community: User-submitted signatures

Import scripts planned in scripts/import_signatures.py

Next Steps

Phase 1: API Development (Weeks 1-2)

# FastAPI endpoints needed:
POST /api/submit         # Upload .sub file + GPS
POST /api/submit/batch   # ZIP + manifest
GET  /api/search         # Query by location
GET  /api/devices/{id}   # Device details
GET  /api/heatmap        # Density data

Phase 2: Web Interface (Weeks 3-4)

• Upload form (drag-and-drop)
• Leaflet.js map with markers
• Device search and filtering
• Statistics dashboard

Phase 3: Signature Import (Weeks 5-6)

• Clone Flipper Zero firmware repo
• Parse and import .sub files
• Extract RTL_433 protocol definitions
• Build device catalog

Phase 4: Community Features (Weeks 7-8)

• User accounts (optional)
• Manual device identification
• Photo uploads
• Voting system

Benefits of Platform Approach

1. Wider Adoption

• ✅ No hardware requirements
• ✅ Works with ANY capture device
• ✅ Lower barrier to entry
• ✅ Focus on data, not devices

2. Community Growth

• ✅ Flipper Zero users (largest community)
• ✅ RTL-SDR enthusiasts
• ✅ Security researchers
• ✅ Ham radio operators

3. Data Quality

• ✅ Signature matching improves over time
• ✅ Community verification
• ✅ Multiple sources = better coverage

4. Simplicity

• ✅ No device drivers or firmware
• ✅ No Android/Termux complexity
• ✅ Just upload and go
• ✅ Platform handles the rest

Migration Notes

Files Removed/Obsoleted

• src/capture/tembed.py - Device-specific controller
• src/gps/ - Android GPS integration (not needed)
• docs/tembed_setup.md - Hardware setup guide (keep for reference)

Files Repurposed

• src/capture/ - Now for file upload handling
• src/gps/ - Now for GPS coordinate validation

New Focus Areas

1. Upload Processing: Handle file uploads efficiently
2. Parsing Pipeline: Robust .sub file parsing
3. Matching Accuracy: Improve signature matching
4. Visualization: Map rendering performance

Success Metrics

Platform Growth

• Unique .sub files submitted
• Geographic coverage (cities/countries)
• Total devices identified
• Community contributors

Matching Accuracy

• Automatic identification rate
• Average confidence score
• Manual verification rate
• Pattern database growth

Community Engagement

• User registrations
• Manual identifications
• Votes cast
• Photo submissions

Summary

GigLez is now a Wigle.net for IoT RF devices - a crowdsourced platform where anyone can:

1. Upload .sub files from any capture device
2. Get automatic device identification via signature matching
3. Visualize IoT device distribution on maps
4. Contribute to community knowledge

No hardware lock-in. No app required. Just upload and explore.

---

See CLAUDE.md for complete technical specifications.