Adds src/api/ingest.py and refactors the live upload handler to auto-detect and route each submission by format instead of assuming .sub: - Flipper .sub -> existing parser + signature matcher (unchanged) - rtl_433 .json/.ndjson -> decoded; model is the device (conf 1.0) - Wigle-style .csv -> decoded; one observation per row (conf 0.9) - .zip batch -> recursed; any mix of the above Also adds stable dedup (SHA256 for whole files, composite key for decoded records) so re-submissions are skipped rather than stored twice, optional GPS privacy rounding via manifest privacy_gps_decimals, a session-level GPS fallback, and helpful errors for unsupported formats. Documented in docs/SUBMISSION_FORMAT.md with rtl_433/CSV sample fixtures. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4.4 KiB
GigLez Submission Format
GigLez accepts Sub-GHz captures from a range of popular tools. Every submission
is a multipart/form-data POST to:
POST /api/v1/captures/upload
with two parts:
| Part | Type | Purpose |
|---|---|---|
files |
one or more file uploads | the captures themselves |
manifest |
JSON string (form field) | GPS, timestamps, session + privacy options |
Format is auto-detected per file (by extension, then by content sniffing), so you can mix types in a single request. Supported formats:
| Format | Extensions | Detected by | Handling |
|---|---|---|---|
| Flipper Zero | .sub |
Filetype: header |
parsed + signature-matched (category router → protocol DB) |
| rtl_433 | .json, .ndjson, .jsonl |
leading { / [ |
already decoded — model is the device |
| Wigle-style tabular | .csv |
comma in first line | already decoded — one row per observation |
| Batch | .zip (PK magic) |
zip magic | recursed; any mix of the above + optional manifest.json |
Manifest
{
"data_source": "production",
"session_uuid": "optional-session-id",
"latitude": 34.0200,
"longitude": -118.3000,
"privacy_gps_decimals": 4,
"captures": [
{
"filename": "capture_001.sub",
"latitude": 34.0201,
"longitude": -118.3011,
"timestamp": "2026-07-18T12:05:00Z"
}
]
}
| Field | Scope | Notes |
|---|---|---|
data_source |
submission | production, test, or mock. Defaults to production. |
session_uuid |
submission | Optional grouping id, stored as session_id. |
latitude / longitude |
submission | Session-level GPS fallback for decoded records that carry no coords. |
privacy_gps_decimals |
submission | If set, all coordinates are rounded to this many decimals before storage. Omit to store full precision. |
captures[] |
per file | Keyed by filename. Supplies per-file GPS + timestamp. |
GPS resolution order
- Coordinates in the record itself (rtl_433 / CSV
latitude/longitude, or GPS embedded in a.subfilename) - Per-file manifest entry (
captures[]matched byfilename) - Session-level manifest GPS (top-level
latitude/longitude)
The chosen source is recorded on each capture as gps_source
(record / manifest / session / manual / none).
Format details
Flipper Zero .sub
Both decoded key files (Protocol / Bit / Key) and RAW pulse captures
(RAW_Data). These are matched through the category router and protocol
database; the result carries match_confidence and match_method.
rtl_433 .json / .ndjson
One JSON object per line (NDJSON) or a single JSON array. rtl_433 output is
already decoded, so the reported model is taken as the device (no signature
matching). Recognised fields include model, id, channel, battery_ok,
temperature_C, humidity, pressure_kPa, mod, freq, time. Frequency in
MHz is normalised to Hz. match_method = rtl_433_decoded, confidence 1.0.
Wigle-style .csv
Header row + one observation per row. Useful columns: model (or
device_name / protocol), manufacturer/brand, frequency (Hz or MHz),
latitude/longitude, timestamp, plus optional decoded fields. match_method
= csv_import, confidence 0.9.
.zip batch
Any mix of the above. Directory entries, dotfiles, and a top-level
manifest.json member are skipped; every other member is ingested by its own
detected format.
Deduplication
Re-submitting the same data is a no-op. Each stored capture carries a stable
dedup_key:
- Whole-file formats (
.sub) — SHA256 of the raw file bytes. - Decoded records (rtl_433 / CSV) — composite of
protocol · device_name · frequency · id · channel · rounded lat/lon · timestamp(to the second).
On upload, records whose key already exists are reported under skipped rather
than stored again.
Response
{
"success": true,
"message": "Processed 9 records (0 duplicates skipped, 0 failed)",
"successful": [ /* stored capture records */ ],
"skipped": [ /* duplicate records, not stored */ ],
"failed": [ { "filename": "...", "error": "..." } ],
"total_uploaded": 2,
"total_successful": 9,
"total_skipped": 0,
"total_failed": 0
}
A single uploaded file (rtl_433/CSV/ZIP) can expand into many capture records, so counts are per-record, not per-file.