Trilltechnician f5d92f1d36 feat: Add wardriving data import system and cleanup tools
Major Features:
- Multi-format GPS data import (Wigle CSV, GPS JSON, filename GPS)
- Data source tracking (test/mock/production)
- Database cleanup tools for managing test data
- JSON file persistence for simplified server
- UI improvements for map controls

Backend:
- Added wardriving_importer.py with WigleCSVImporter, GPSJSONImporter, BatchImporter
- Added data_source and session_id tracking to captures
- New API endpoints: DELETE /api/v1/admin/cleanup
- Auto-save/load functionality for captures_simple.json
- Updated stats endpoint to show data source breakdown

CLI Tools:
- scripts/import_wardriving_data.py - Batch import with --data-source flag
- scripts/cleanup_database.py - Clean by source, session, or all
- scripts/create_test_dataset.py - Generate GPS-tagged test data
- scripts/test_wardriving_import.py - Test suite for importers

Frontend:
- Fixed map controls z-index and positioning issues
- Moved controls to top-right to avoid zoom button overlap
- Fixed Leaflet zoom controls rendering over header
- Changed controls to position:fixed for persistent visibility
- Export map object to window.map for proper invalidateSize

Documentation:
- docs/WARDRIVING_IMPORT.md - Complete import guide
- docs/DATA_CLEANUP_GUIDE.md - Cleanup system documentation
- docs/TEST_LOCATIONS.md - Test GPS coordinates reference
- TEST_RESULTS_SUMMARY.md - Format testing results

Test Data:
- Created test_dataset_gps with 15 captures across 5 US cities
- All test data marked with data_source="test" for easy cleanup

Testing:
- Verified Wigle CSV import (1 capture from West LA)
- Verified GPS filename import (3 captures from UCLA area)
- Verified GPS JSON companion files (15 captures, 5 cities)
- Verified cleanup functionality (deleted 15 test captures)
- Verified data persistence across server restarts

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-14 07:48:01 -08:00
2026-01-12 18:21:11 -08:00
2026-01-12 18:21:11 -08:00

GigLez - IoT RF Device Mapping Platform

A Wigle.net-inspired crowdsourced platform for mapping and identifying Sub-GHz IoT RF devices.

Overview

GigLez is a platform-agnostic web service that accepts .sub/.fff file uploads with GPS coordinates, automatically identifies IoT devices using signature databases, and visualizes device distribution on interactive maps. Think of it as Wigle.net for RF IoT devices instead of WiFi networks.

Key Principle: We don't care what hardware you use to capture signals. If you can generate .sub files with GPS coordinates, you can contribute to the platform.

Features

Core Platform Features

  • File Upload: Submit .sub/.fff files with GPS coordinates via web or API
  • Automatic Parsing: Extract frequency, protocol, modulation, and timing from files
  • Device Identification: Match against 1000+ known signatures (Flipper Zero, RTL_433)
  • Interactive Maps: Visualize captured devices with heatmaps and clustering
  • Search & Filter: Query by location, device type, frequency, protocol
  • Anonymous Uploads: No account required (but optional for tracking contributions)

Community Features

  • Manual Identification: Add or correct device IDs with photo evidence
  • Voting System: Upvote/downvote community identifications
  • Verification: Earn reputation for accurate identifications
  • Leaderboards: Track top contributors
  • Data Export: Download captures as .sub, JSON, CSV, or GeoJSON

How It Works

┌─────────────────┐
│  Capture Device │  (Flipper Zero, LilyGo, RTL-SDR, HackRF, etc.)
│  + GPS Source   │
└────────┬────────┘
         │
         ↓ .sub files + GPS coords
┌─────────────────┐
│  GigLez Upload  │  (Web form or API)
└────────┬────────┘
         │
         ↓ Parse & Match
┌─────────────────┐
│  Device ID      │  (Automatic matching: "Chamberlain Garage Opener")
│  Confidence: 95%│
└────────┬────────┘
         │
         ↓ Store & Display
┌─────────────────┐
│  Interactive    │
│  Map View       │
└─────────────────┘

Supported Capture Devices

GigLez accepts .sub files from any capture device:

  • Flipper Zero - Native .sub format
  • LilyGo T-Embed - Compatible with Bruce/Marauder firmware
  • HackRF One - Convert captures to .sub format
  • RTL-SDR - Use rtl_433 + conversion tools
  • YardStick One - Convert RfCat captures
  • Custom Solutions - Any tool that outputs .sub/.fff format

Don't have a capture device? You can still browse and search the community database!

Quick Start

Submit Your First Capture

Via Web Interface

  1. Visit https://giglez.io/upload
  2. Drag & drop your .sub files
  3. Enter GPS coordinates (or upload CSV manifest)
  4. (Optional) Create account to track submissions
  5. Click Submit - automatic device matching begins!

Via API

curl -X POST https://api.giglez.io/submit \
  -H "Content-Type: multipart/form-data" \
  -F "file=@capture_001.sub" \
  -F "latitude=40.7128" \
  -F "longitude=-74.0060" \
  -F "timestamp=2025-01-11T20:30:00Z"

Batch Upload

# Create manifest.json
{
  "captures": [
    {
      "filename": "capture_001.sub",
      "latitude": 40.7128,
      "longitude": -74.0060,
      "timestamp": "2025-01-11T20:30:00Z"
    }
  ]
}

# Upload ZIP file
curl -X POST https://api.giglez.io/submit/batch \
  -F "manifest=@manifest.json" \
  -F "archive=@captures.zip"

Submission Format

Required Fields

  • GPS Coordinates: Latitude/Longitude (decimal degrees)
  • Timestamp: ISO 8601 format (e.g., 2025-01-11T20:30:00Z)
  • .sub/.fff File: Signal capture file

Optional Fields

  • Altitude: Meters above sea level
  • Accuracy: GPS accuracy in meters
  • Device Name: What you used to capture (e.g., "Flipper Zero")
  • Notes: Additional context
  • Photos: Device identification evidence

Example .sub File

Filetype: Flipper SubGhz Key File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok270Async
Protocol: Princeton
Bit: 24
Key: 00 00 00 00 00 95 D5 D4
TE: 400

Device Identification

Automatic Matching

GigLez uses a multi-strategy matching engine:

  1. Exact Match (100% confidence): Protocol + Frequency + Bit Length
  2. Partial Match (80% confidence): Protocol + Frequency
  3. Pattern Match (70-90% confidence): Bit pattern similarity
  4. Timing Match (60-80% confidence): Pulse timing characteristics
  5. Frequency Match (50-70% confidence): Frequency proximity

Signature Databases

  • Flipper Zero Database: 1000+ device signatures (.sub files)
  • RTL_433 Protocols: 200+ protocol definitions
  • Community Signatures: User-submitted verified devices

Manual Identification

If automatic matching fails or is low confidence, users can:

  • Submit device identification with photos
  • Vote on other users' identifications
  • Earn reputation for verified IDs

API Documentation

Authentication

# Get API token (optional, for tracking submissions)
curl -X POST https://api.giglez.io/auth/register \
  -d "email=user@example.com" \
  -d "username=wardriver"

# Use token in requests
curl -H "Authorization: Bearer YOUR_TOKEN" \
  https://api.giglez.io/api/submit

Endpoints

Endpoint Method Description
/api/submit POST Upload single capture
/api/submit/batch POST Upload multiple captures
/api/search GET Search captures by location/device
/api/devices GET Browse device catalog
/api/devices/{id} GET Device details and captures
/api/heatmap GET Geographic density data
/api/stats GET Platform statistics

See full documentation at https://api.giglez.io/docs

Privacy & Security

GPS Anonymization

  • Precision Control: Round coordinates to desired precision (default: 10m)
  • Private Mode: Opt-out of public database
  • Anonymous Uploads: No account required

Data Removal

Request removal of your submissions:

curl -X DELETE https://api.giglez.io/api/captures/{id} \
  -H "Authorization: Bearer YOUR_TOKEN"

No PII Collection

  • .sub files contain no personally identifiable information
  • GPS coordinates are approximate (not exact addresses)
  • Optional accounts for contribution tracking only

Development Setup

Prerequisites

# Install PostgreSQL
sudo apt install postgresql postgresql-contrib postgis

# Install Python 3.10+
sudo apt install python3.10 python3-pip

Installation

# Clone repository
git clone https://github.com/yourusername/giglez.git
cd giglez

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Initialize database
python scripts/init_db.py

# Import signature databases
python scripts/import_signatures.py

# Run development server
uvicorn src.api.main:app --reload

Project Structure

giglez/
├── src/
│   ├── parser/          # .sub file parsing
│   ├── matcher/         # Device signature matching
│   ├── database/        # PostgreSQL models
│   ├── api/             # FastAPI endpoints
│   └── web/             # Web interface
├── signatures/          # Known device signatures
│   ├── flipper/         # Flipper Zero .sub files
│   ├── rtl433/          # RTL_433 protocols
│   └── community/       # User submissions
├── docs/                # Documentation
└── tests/               # Test suite

Wigle.net Comparison

Feature Wigle.net GigLez
Data Type WiFi, Bluetooth, Cellular Sub-GHz IoT RF (300-928 MHz)
Upload Format CSV .sub/.fff files + GPS
Identification MAC/SSID (exact) Signature matching (fuzzy)
Scale 349M+ networks Just getting started!
Focus Network mapping Device type identification
Privacy Public by default Opt-in sharing

Contributing

We welcome contributions! See CONTRIBUTING.md

Ways to Contribute

  • 📡 Upload Captures: Share your .sub files with GPS
  • 🔍 Identify Devices: Add manual IDs with photos
  • 🛠️ Add Signatures: Contribute protocol definitions
  • 💻 Code: Improve matching algorithms, UI, API
  • 📖 Documentation: Tutorials, guides, translations

Roadmap

  • Platform architecture & database design
  • .sub file parser
  • Signature matching engine
  • Web upload interface
  • Interactive map visualization
  • API v1 release
  • Mobile app (Android/iOS)
  • Real-time collaboration features
  • Protocol decoder for unknown signals

License

MIT License - see LICENSE

Acknowledgments

Inspired by:

  • Wigle.net - WiFi/cellular mapping platform
  • Flipper Zero - Sub-GHz signature database
  • RTL_433 - Protocol definitions
  • The wardriving and RF security community

Support


⚠️ Legal Notice: This tool is for research and educational purposes. Always comply with local radio frequency regulations. Capturing RF signals may be regulated in your jurisdiction. GigLez is for passive monitoring only.

S
Description
GigLez - Sub-GHz RF IoT device mapping platform (Wigle for IoT)
Readme 16 MiB
Languages
Python 78%
JavaScript 12.1%
Shell 3.5%
HTML 3.2%
PLpgSQL 2%
Other 1.2%