Trilltechnician/giglez
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: multi-format capture ingestion (rtl_433, CSV, ZIP) + dedup

Browse Source 
Adds src/api/ingest.py and refactors the live upload handler to auto-detect
and route each submission by format instead of assuming .sub:

  - Flipper .sub          -> existing parser + signature matcher (unchanged)
  - rtl_433 .json/.ndjson -> decoded; model is the device (conf 1.0)
  - Wigle-style .csv      -> decoded; one observation per row (conf 0.9)
  - .zip batch            -> recursed; any mix of the above

Also adds stable dedup (SHA256 for whole files, composite key for decoded
records) so re-submissions are skipped rather than stored twice, optional
GPS privacy rounding via manifest privacy_gps_decimals, a session-level GPS
fallback, and helpful errors for unsupported formats. Documented in
docs/SUBMISSION_FORMAT.md with rtl_433/CSV sample fixtures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
leetcrypt
2026-07-18 20:08:39 -07:00
parent 34b08c4ad6
commit 11d8957a49
5 changed files with 585 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/fixtures/ingest_samples/rtl433_sample.ndjson
Vendored
+5
View File
@@ -0,0 +1,5 @@
{"time":"2026-07-18 12:01:03","model":"Acurite-Tower","id":11524,"channel":"A","battery_ok":1,"temperature_C":22.4,"humidity":48,"mod":"ASK","freq":433.92}
{"time":"2026-07-18 12:01:07","model":"Schrader-TPMS","type":"TPMS","id":"7a3f01","pressure_kPa":221.0,"temperature_C":19.0,"mod":"FSK","freq":315.0}
{"time":"2026-07-18 12:01:15","model":"Honeywell-Security","id":204512,"channel":0,"state":"open","mod":"ASK","freq":345.0}
{"time":"2026-07-18 12:01:22","model":"LaCrosse-TX141THBv2","id":227,"channel":1,"battery_ok":1,"temperature_C":24.1,"humidity":52,"mod":"ASK","freq":433.92}
{"time":"2026-07-18 12:01:31","model":"Efergy-e2","id":8210,"power_W":812,"mod":"ASK","freq":433.55}
tests/fixtures/ingest_samples/wigle_sample.csv
Vendored
+5
View File
@@ -0,0 +1,5 @@
model,manufacturer,frequency,latitude,longitude,timestamp,id,channel
Chamberlain-Garage,Chamberlain,390000000,34.0201,-118.3011,2026-07-18T12:05:00Z,4471,
Nexus-TH,Nexus,433920000,34.0215,-118.2994,2026-07-18T12:06:10Z,88,1
Ford-TPMS,Ford,315000000,34.0189,-118.3040,2026-07-18T12:07:20Z,a19c02,
Generic-Doorbell,,433920000,34.0230,-118.2971,2026-07-18T12:08:00Z,,
1 model manufacturer frequency latitude longitude timestamp id channel
2 Chamberlain-Garage Chamberlain 390000000 34.0201 -118.3011 2026-07-18T12:05:00Z 4471
3 Nexus-TH Nexus 433920000 34.0215 -118.2994 2026-07-18T12:06:10Z 88 1
4 Ford-TPMS Ford 315000000 34.0189 -118.3040 2026-07-18T12:07:20Z a19c02
5 Generic-Doorbell 433920000 34.0230 -118.2971 2026-07-18T12:08:00Z