giglez/docs/WARDRIVER_ONBOARDING.md

GigLez - RF Wardriver Onboarding & Data Format Integration

Welcome, RF Wardrivers! This document explains what we've built, current system capabilities, and critically, what data formats we need to support for your existing wardriving workflows.

---

What We've Built: GigLez Overview

GigLez is a Wigle-style crowdsourced platform for mapping Sub-GHz IoT RF devices (300-928 MHz). Think "Wigle.net but for garage doors, weather stations, tire pressure monitors, and doorbells instead of WiFi."

Core Concept

• Upload .sub files (Flipper Zero format) with GPS coordinates
• Automatic Device Identification using RF signature matching (299 protocol database)
• Interactive Map showing IoT device distribution by location
• Community Verification for device IDs and crowdsourced improvements

---

What's Working Right Now

✅ Device Identification System (6-Component Scoring)

We've built a multi-factor RF signature matching engine with 33% accuracy on synthetic signals (baseline established):

Component	Weight	Purpose
Timing Analysis	40%	K-means clustering to extract SHORT/LONG pulse widths
Preamble Detection	25%	Identifies sync patterns (alternating, long_burst, sync_word)
Timing Ratio	20%	Discriminates protocols by LONG/SHORT ratio (2:1 vs 3:1)
Frequency Fingerprinting	10%	ISM band classification (315/433/868/915 MHz, ±100kHz tolerance)
Bit Count Matching	5%	Validates bit length against protocol expectations

Processing Speed: 156ms average per signal (0.4ms parse + 156ms matching)

Supported Modulations: OOK, FSK (from .sub metadata)

✅ Protocol Database (299 Protocols)

Imported from RTL_433 project:

• Weather Sensors: LaCrosse, Oregon Scientific, Acurite, Nexus (70+ protocols)
• Garage Doors: Chamberlain, LiftMaster, Linear (15+ protocols)
• TPMS: Schrader, Toyota, Honda, Ford (30+ protocols)
• Security: Honeywell, SimpliSafe, DSC (20+ protocols)
• Doorbells & Remotes: Princeton, PT2262, EV1527 (40+ protocols)
• Lighting/Climate: LED controllers, ceiling fans, thermostats (20+ protocols)

Frequency Coverage:

• 315 MHz: 15% (garage doors, car keys)
• 433 MHz: 70% (weather, doorbells, generic remotes)
• 868 MHz: 10% (EU devices)
• 915 MHz: 5% (US ISM band)

✅ GPS Validation & Privacy Features

Implemented (src/gps/validator.py):

• Latitude/longitude validation (-90 to 90, -180 to 180)
• Coordinate precision rounding (configurable 4-8 decimal places)
• Altitude validation (optional, -500m to 10000m)
• Haversine distance calculation for duplicate detection
• Privacy: GPS anonymization via coordinate rounding

Example:

from src.gps.validator import validate_coordinates, anonymize_location

# Validate raw GPS
coords = validate_coordinates(lat=40.712776, lon=-74.005974, altitude=10.5)

# Anonymize (round to ~11m precision)
anon = anonymize_location(lat=40.712776, lon=-74.005974, precision=4)
# Returns: (40.7128, -74.0060)

✅ File Format Parsing (.sub files)

Currently Supported: Flipper Zero .sub format (both RAW and decoded variants)

Parser (src/parser/sub_parser.py):

• Extracts frequency, preset (modulation), protocol name
• Parses RAW_Data into pulse sequences
• Handles both OOK and FSK modulation metadata
• Validates file structure

Example .sub file:

Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok650Async
Protocol: RAW
RAW_Data: 2980 -240 520 -980 520 -980 1000 -500 ...

⚠️ Test Coverage

• Unit Tests: 56 tests passing (timing, preamble, frequency, GPS, database)
• Synthetic Benchmark: 33% top-1 accuracy on 12 test signals
• Real-World Benchmark: 0% accuracy on 11 real Flipper captures ❌

---

Critical Gap: Real-World Performance Issue

The Problem: Synthetic vs Real

We optimized for synthetic test signals, but real Flipper Zero community captures fail completely.

Root Causes (analyzed in docs/REAL_CAPTURE_ANALYSIS.md):

1. Decoded vs RAW Format (45% of failures)

   • Many real .sub files are already decoded (Protocol: Holtek_HT12X)
   • Our system only processes RAW_Data fields
   • Fix Needed: Parse decoded .sub files by protocol name + timing element (TE)

2. Missing Protocols (27% of failures)

   • Database lacks: Acurite 02077M, GE Doorbell 19297, Byron DB421E, Holtek HT12X
   • Fix Needed: Import from real captures or community contributions

3. Signal Complexity (27% of failures)

   • Real captures have multi-packet transmissions (131 pulses vs 40 bits)
   • Variable signal quality, interference beyond 15% jitter tolerance
   • Fix Needed: Packet segmentation, higher noise tolerance (15% → 25%)

4. Protocol Family Competition (9% of failures)

   • Nexus ranks #34, beaten by similar Oregon Scientific protocols
   • Fix Needed: Protocol family grouping and boosting

---

What We Need From You: Data Format Questions

To make GigLez seamless for existing RF wardriving workflows, we need to understand your data collection methods and formats.

1. Hardware & Capture Tools

What RF capture hardware do you use?

• Flipper Zero
• RTL-SDR dongles
• HackRF One
• LimeSDR
• YARD Stick One
• LilyGo T-Watch / LoRa devices
• Other: _______________

What software do you use to capture signals?

• Flipper Zero firmware (generates .sub files)
• rtl_433 (outputs JSON/CSV)
• Universal Radio Hacker (URH) - saves .complex/.cfile
• GNU Radio (custom flowgraphs)
• inspectrum (visual analysis)
• SDR# (saves .wav / IQ recordings)
• gqrx (IQ recordings)
• Other: _______________

2. File Format Preferences

What file formats do your captures produce?

• .sub (Flipper Zero)
• .fff (Flipper Zero older format)
• JSON (rtl_433 output)
• CSV (rtl_433 CSV mode)
• .complex / .cfile (URH raw IQ data)
• .wav (SDR# IQ recordings)
• .sigmf (SigMF metadata format)
• .cu8 / .cs8 (raw IQ samples)
• Other: _______________

Would you prefer to upload:

• Individual files with GPS metadata embedded
• Batch uploads (ZIP with manifest.json)
• Real-time streaming (live wardriving session)
• CSV/JSON logs referencing stored signal files

3. GPS Data Association

How do you currently associate GPS coordinates with captures?

• Manual entry (lat/lon typed per file)
• GPX track log synced by timestamp
• NMEA GPS logs (separate file)
• Embedded in capture file metadata
• Kismet-style CSV (BSSID, lat, lon, timestamp)
• Custom database/spreadsheet
• Other: _______________

GPS Precision Requirements:

• What precision do you typically capture? (e.g., 6 decimal places = ±0.11m)
• Do you need altitude data stored?
• Do you want GPS anonymization options (round to city-level precision)?

4. Metadata & Session Context

What additional metadata do you typically log?

• Device photos (visual identification)
• Signal strength (RSSI)
• Capture duration
• Antenna type/gain
• Weather conditions
• Session notes/comments
• Device orientation (N/S/E/W)
• Other: _______________

5. Batch Upload Workflows

Typical wardriving session size:

• How many captures per session? (e.g., 50, 500, 5000+)
• Do you prefer uploading during the drive or post-processing later?
• Would you use a CLI tool for bulk uploads or web interface?

Preferred batch format:

Option A: ZIP with manifest.json
captures.zip
├── capture_001.sub
├── capture_002.sub
├── ...
└── manifest.json  # GPS + timestamps for all files

Option B: CSV index + separate signal files
session_2025-01-15.csv (columns: filename, lat, lon, timestamp, notes)
signals/
├── capture_001.sub
├── capture_002.sub
└── ...

Option C: Real-time API streaming (JSON payloads)

Which format would integrate best with your existing tools?

6. Device Identification Preferences

When you capture unknown signals, what info helps you ID them?

• Frequency alone (e.g., "433.92 MHz probably garage door")
• Timing patterns (pulse widths)
• Bit patterns (binary data)
• Physical device photo
• FCC ID database lookup
• Community voting/verification
• Other: _______________

Confidence threshold for auto-tagging:

• Would you trust automatic IDs at 80% confidence?
• Would you manually verify all IDs regardless of confidence?
• Prefer "suggested IDs" that you confirm before publishing?

7. Privacy & Data Sharing

Data sharing preferences:

• Make all captures public (Wigle-style open database)
• Private by default (opt-in sharing)
• Share anonymized location data only (round to city)
• Share device types but not GPS coordinates
• Custom privacy controls per capture

Removal requests:

• Should users be able to request removal of specific captures (like Wigle's BSSID opt-out)?

---

Integration Roadmap

Based on your feedback, we'll prioritize:

Phase 1: Core Format Support

• rtl_433 JSON import (most common for non-Flipper wardrivers)
• Decoded .sub file support (fix 45% of current failures)
• GPX track log parsing

Phase 2: Batch Upload System

• ZIP + manifest.json uploader
• CLI tool for bulk submissions
• API endpoints for real-time streaming

Phase 3: Additional Hardware Support

• URH .complex/.cfile import (convert to timing patterns)
• SDR# .wav IQ file processing
• SigMF metadata parsing

Phase 4: Community Features

• Manual device ID submission
• Photo upload for visual verification
• Voting system for ID accuracy
• Protocol signature contributions

---

How to Contribute Your Data Now

Option 1: Share Sample Files

Send us 5-10 representative captures from your wardriving sessions:

• Include GPS coordinates (lat/lon/timestamp)
• Mix of known and unknown devices
• Note your capture tool and workflow

Where to send: Create GitHub issue or email

Option 2: Test the Current System

Upload Flipper Zero .sub files via our API (coming soon):

curl -X POST https://giglez.io/api/submit \
  -F "file=@capture.sub" \
  -F "latitude=40.7128" \
  -F "longitude=-74.0060" \
  -F "timestamp=2025-01-15T12:00:00Z"

Option 3: Provide rtl_433 JSON

If you use rtl_433, share sample JSON output:

rtl_433 -F json > captures.json
# Include GPS coordinates for each capture (manual or GPX sync)

---

Questions?

We want to build this for the wardriving community, so your input directly shapes development priorities.

Contact:

• GitHub Issues: giglez/issues
• Email: team@giglez.io (placeholder)
• Discord: GigLez Community (placeholder)

---

Response Template (Copy/Paste & Fill Out)

### My Wardriving Setup

**Hardware**: [e.g., HackRF One + GPS dongle]
**Software**: [e.g., rtl_433 + GPX logger]
**File Formats**: [e.g., JSON output from rtl_433]
**GPS Method**: [e.g., GPX track synced by timestamp]
**Session Size**: [e.g., 200-500 captures per drive]
**Upload Preference**: [e.g., Batch upload via CLI after session]

### Data Format Details

[Paste sample file snippet or describe structure]

### Integration Requests

[What would make GigLez fit seamlessly into your workflow?]

### Privacy Preferences

[Public/private/anonymized?]

---

Thank you for helping build the Sub-GHz IoT mapping platform! 🚗📡

Last Updated: 2026-02-16