giglez/docs/WARDRIVING_IMPORT.md

# Wardriving Data Import Guide

## Overview

GigLez now supports importing GPS-tagged SubGhz captures from popular wardriving formats used by Flipper Zero, Bruce firmware, and other open-source SubGhz tools.

## Supported Formats

### 1. Wigle-WiFi CSV (Adapted for SubGhz)

**Format**: Based on Wigle.net WiFi wardriving CSV format, adapted for SubGhz signals

**Example**: `signatures/t-embed-rf/test_export_wigle.csv`

```csv
WigleWifi-1.4,appRelease=SubGHz-Wardrive-1.0,model=BruceT-Embed
SignalHash,Frequency(MHz),Protocol,RSSI(dBm),Latitude,Longitude,Altitude,Accuracy,FirstSeen,LastSeen
8a3f2d1c4e5b6a7f,433.920,RAW,,34.073625,-118.359557,,24.5,2026-01-13T19:17:51Z,2026-01-13T19:17:51Z
```

**Fields**:
- `SignalHash`: Unique identifier for the signal
- `Frequency(MHz)`: Frequency in MHz
- `Protocol`: Detected protocol (RAW, Princeton, etc.)
- `RSSI(dBm)`: Signal strength (optional)
- `Latitude`, `Longitude`: GPS coordinates (required)
- `Altitude`, `Accuracy`: GPS metadata (optional)
- `FirstSeen`, `LastSeen`: Timestamps
- `BruceFile`: Associated .sub filename
- `SessionID`: Wardriving session identifier

### 2. GPS Coordinate JSON

**Format**: Standalone GPS coordinate files

**Example**: `signatures/t-embed-rf/gps_coordinates_20260109_212651.json`

```json
{
  "type": "gps_coordinate",
  "source": "phone_wardriving",
  "data": {
    "latitude": 34.0522,
    "longitude": -118.2437,
    "accuracy": 5.0,
    "altitude": 100.0,
    "timestamp": "2026-01-09T21:26:51Z"
  },
  "session": "subghz_wardriving"
}
```

**Usage**: Place JSON file with same name as .sub file:
- `capture_001.sub` → `capture_001.json`

### 3. GPS in Filenames

**Format**: Embedded GPS coordinates in filename

**Examples**:
- `34.0478N_118.2348W_1637_raw_8.sub` → 34.0478, -118.2348
- `lat34.0478lon-118.2348_capture.sub` → 34.0478, -118.2348
- `-34.0478_118.2348_test.sub` → -34.0478, 118.2348

**Supported Patterns**:
1. N/S/E/W: `LAT[NS]_LON[EW]`
2. lat/lon prefix: `latLATlonLON`
3. Signed decimal: `LAT_LON`

## Import Tools

### CLI Import Tool

**Location**: `scripts/import_wardriving_data.py`

**Usage**:

```bash
# Import from Wigle CSV
./scripts/import_wardriving_data.py signatures/t-embed-rf/test_export_wigle.csv

# Import from directory with GPS in filenames
./scripts/import_wardriving_data.py signatures/t-embed-rf/

# Dry run (parse but don't upload)
./scripts/import_wardriving_data.py test_export.csv --dry-run

# Specify API endpoint
./scripts/import_wardriving_data.py data/ --api-url http://localhost:8000
```

**Options**:
- `source`: CSV file or directory containing .sub files
- `--csv`: CSV manifest file (for directory imports)
- `--api-url`: GigLez API URL (default: http://localhost:8000)
- `--dry-run`: Parse files but do not upload

### Python API

```python
from src.parser.wardriving_importer import (
    WigleCSVImporter,
    BatchImporter,
    GPSJSONImporter
)

# Import from Wigle CSV
csv_importer = WigleCSVImporter()
captures = csv_importer.parse_csv("test_export_wigle.csv")

# Import from directory
batch_importer = BatchImporter()
captures = batch_importer.import_from_directory("captures/")

# Generate upload manifest
manifest = batch_importer.to_upload_manifest()
```

## GPS Priority System

When importing .sub files, GPS coordinates are extracted in this priority:

1. **CSV Manifest** (if provided) - Highest priority
2. **Companion JSON file** - e.g., `capture.json` for `capture.sub`
3. **GPS in filename** - Auto-extracted from filename pattern
4. **Skip file** - If no GPS source found

## Examples

### Example 1: Import Wigle CSV from Bruce Firmware

```bash
# Bruce firmware exports SubGhz wardriving data to Wigle CSV format
./scripts/import_wardriving_data.py ~/SD_Card/SubGhz/wardrive_export.csv
```

**Output**:
```
============================================================
Importing from Wigle CSV
============================================================
File: wardrive_export.csv

✅ Parsed 15 captures

Sample capture:
  Filename: raw_433920_001.sub
  Frequency: 433.92 MHz
  GPS: 34.073625, -118.359557
  Accuracy: 24.5m

📤 Uploading 15 captures to http://localhost:8000

[1/15] Uploading raw_433920_001.sub...
  ✅ Success
...

============================================================
Upload Complete:
  Successful: 15
  Failed: 0
  Total: 15
============================================================
```

### Example 2: Import Directory with GPS in Filenames

```bash
# T-Embed captures with GPS embedded in filenames
./scripts/import_wardriving_data.py ~/RF_Captures/tembed/
```

**Output**:
```
============================================================
Importing from Directory
============================================================
Directory: ~/RF_Captures/tembed/

✅ Found 8 captures with GPS

GPS Sources:
  filename_nsew: 8

📤 Uploading 8 captures to http://localhost:8000
...
```

### Example 3: Import with Companion JSON Files

**Directory Structure**:
```
captures/
  capture_001.sub
  capture_001.json  ← GPS data
  capture_002.sub
  capture_002.json  ← GPS data
```

**Command**:
```bash
./scripts/import_wardriving_data.py captures/
```

**Result**: GPS automatically loaded from companion JSON files

## Verification

After importing, verify data in web interface:

1. **Map View**: http://localhost:8000
   - All imported captures appear as markers
   - Color-coded by frequency band

2. **Statistics**: View upload counts
   ```bash
   curl http://localhost:8000/api/v1/stats/summary | jq
   ```

3. **Query API**: Search imported data
   ```bash
   curl http://localhost:8000/api/v1/query/captures | jq
   ```

## Supported Devices

### Tested Formats

| Device | Format | GPS Source | Status |
|--------|--------|------------|--------|
| **Bruce T-Embed** | Wigle CSV | Phone GPS via Bluetooth | ✅ Working |
| **Flipper Zero** | .sub filename | Manual GPS entry | ✅ Working |
| **Flipper + GPS Module** | Companion JSON | GPS module | ✅ Working |
| **RTL-SDR** | .sub + CSV | Software GPS | ✅ Working |

### Bruce Firmware SubGhz Wardriving

**Setup**:
1. Install Bruce firmware on T-Embed or similar device
2. Pair phone via Bluetooth for GPS
3. Run SubGhz wardriving mode
4. Export to Wigle CSV format

**Export Format**: Compatible with GigLez importer

**GPS Accuracy**: Typically 5-25m (from phone GPS)

### Flipper Zero with GPS

**Option 1**: GPS in Filename
- Manually name files with coordinates
- Format: `34.0478N_118.2348W_capture.sub`

**Option 2**: GPS Module
- Use Flipper GPS backpack/module
- Export coordinates to companion JSON
- Place JSON alongside .sub file

**Option 3**: Wardriving App
- Use community wardriving apps
- Export to Wigle CSV format

## Testing

**Test Suite**: `scripts/test_wardriving_import.py`

```bash
# Run all import tests
./scripts/test_wardriving_import.py
```

**Tests**:
1. Wigle CSV parsing
2. GPS JSON parsing
3. Batch directory import
4. Upload manifest generation

## Troubleshooting

### Issue: No GPS Found

**Symptom**: Files skipped during import

**Solutions**:
1. Check filename format matches supported patterns
2. Verify companion JSON files exist and are valid
3. Provide CSV manifest with GPS data

### Issue: Import Fails

**Symptom**: Upload errors during import

**Solutions**:
1. Verify server is running: `curl http://localhost:8000/health`
2. Check .sub file format (must have valid Flipper headers)
3. Try `--dry-run` to test parsing without upload

### Issue: Wrong GPS Coordinates

**Symptom**: Markers appear in wrong location

**Solutions**:
1. Verify N/S/E/W directions in filename
2. Check lat/lon order (latitude first, longitude second)
3. Validate coordinate ranges (-90 to 90 lat, -180 to 180 lon)

## Future Enhancements

### Planned Features

1. **GPX Track Import**
   - Import GPS tracks from wardriving sessions
   - Interpolate positions for captures

2. **KML Export**
   - Export captures to Google Earth format
   - Heatmap visualization

3. **Batch Upload from Phone**
   - Direct upload from Android/iOS app
   - Real-time GPS streaming

4. **Duplicate Detection**
   - Detect and merge duplicate captures
   - Improve accuracy with multiple observations

## Related Documentation

- [GPS Auto-Extraction](GPS_AUTO_EXTRACTION.md) - Filename GPS extraction
- [Web Interface](WEB_INTERFACE_README.md) - Upload interface guide
- [API Documentation](../src/api/README.md) - Upload API details

## Summary

GigLez now supports importing wardriving data from:
- ✅ Wigle CSV format (Bruce firmware, custom apps)
- ✅ GPS coordinate JSON files
- ✅ GPS-embedded filenames
- ✅ Batch directory imports

**Import your wardriving data and visualize SubGhz signals on the map!** 🗺️📡