Trilltechnician/giglez
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Add comprehensive RF test databases guide and download script

Browse Source 
- Created RF_TEST_DATABASES.md with 6 primary databases (30,000+ files)
- Added RTL433_TESTING_GUIDE.md explaining 0% decode rate
- Created download_rf_test_datasets.sh to fetch all test data
- Documented web search results from Google dorking
- Listed repositories: rtl_433_tests, FlipperZero-Subghz-DB, UberGuidoZ, etc
- Included testing strategy and expected success rates
This commit is contained in:
priestlypython
2026-01-14 18:18:31 -08:00
parent 60c07647aa
commit 8842240168
3 changed files with 1082 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/RF_TEST_DATABASES.md
+512
View File
@@ -0,0 +1,512 @@
# RF Signal Test Databases - Comprehensive Guide
## Summary
This document catalogs labeled RF signal databases found via Google dorking and GitHub searches for testing our RTL_433 integration implementation.
**Total Databases Found:** 6 primary + 10+ secondary sources
**Total Estimated Files:** 30,000+ .sub files, 1,000+ .cu8 files
**Best for Testing:** rtl_433_tests (100% labeled, known protocols)
---
## Primary Databases (Recommended)
### 1. RTL_433 Official Test Files ⭐⭐⭐⭐⭐
**Repository:** https://github.com/merbanan/rtl_433_tests
**Quality:** Highest (100% labeled, protocol-verified)
**Format:** `.cu8`, `.json`
**Size:** ~1,000 test files
**Coverage:** 244 protocols
#### Features:
- Official regression test suite for rtl_433
- Each file has corresponding `.json` with expected decode results
- Organized by manufacturer/device type
- Guaranteed to decode with RTL_433
#### Directory Structure:
```
tests/
├── acurite/
│ ├── 01/ # Acurite 606TX
│ ├── 02/ # Acurite Tower
│ └── ...
├── oregon_scientific/
│ ├── 01/ # Oregon v2.1
│ ├── 02/ # Oregon v3
│ └── ...
├── lacrosse/
├── tpms/
│ ├── ford/
│ ├── toyota/
│ └── ...
└── ...
```
#### Protocols Included:
- Weather Sensors: Acurite, Oregon, LaCrosse, Nexus
- TPMS: Ford, Toyota, GM, Schrader, Continental
- Doorbells: Byron, Honeywell
- Security: Visonic, DSC, Paradox
- And 200+ more
#### Testing Priority: **Highest**
**Problem:** Files are in `.cu8` format (complex I/Q samples), not `.sub` format
**Solution:** Need to create converter (see section below)
---
### 2. FlipperZero-Subghz-DB ⭐⭐⭐☆☆
**Repository:** https://github.com/Zero-Sploit/FlipperZero-Subghz-DB
**Quality:** Medium (mixed protocols, community-contributed)
**Format:** `.sub` (Flipper format - ready to use!)
**Size:** 13,717 .sub files, 721 folders
**Coverage:** Mostly garage/gate openers, some sensors
#### Features:
- Largest single collection of .sub files
- Organized by device type
- Ready to use (no conversion needed)
- Community-maintained
#### Directory Structure:
```
Sub-GHz/
├── Garages/
├── Gates/
├── Doorbells/
├── Car_Keys/
├── Weather_Stations/ ← Useful for our testing
├── TPMS/ ← Useful for our testing
└── ...
```
#### Testing Priority: **Medium**
**Pros:** Ready-to-use .sub format
**Cons:** Mostly proprietary protocols (low RTL_433 match rate expected)
---
### 3. UberGuidoZ Flipper Database ⭐⭐⭐⭐☆
**Repository:** https://github.com/UberGuidoZ/Flipper
**Quality:** High (well-organized, actively maintained)
**Format:** `.sub`, `.ir`, `.nfc`, `.bin`
**Size:** ~500MB (10,000+ files)
**Coverage:** Comprehensive multi-protocol collection
#### Features:
- Most comprehensive Flipper Zero resource
- Frequent updates
- Well-documented
- Includes sub-directories for specific devices
#### Notable Subdirectories:
```
Flipper/
├── Sub-GHz/
│ ├── Weather_Sensors/
│ ├── Remotes/
│ ├── Vehicles/
│ └── ...
├── Infrared/
├── NFC/
└── BadUSB/
```
#### Testing Priority: **High**
**Best Use:** Search for specific manufacturer names (Acurite, Oregon, LaCrosse)
---
### 4. Full Flipper Database ⭐⭐⭐☆☆
**Repository:** https://github.com/DefinetlyNotAI/Full_Flipper_Database
**Quality:** Medium (comprehensive but mixed quality)
**Format:** `.sub`, `.ir`, `.nfc`, etc
**Size:** Large (exact count unknown)
**Coverage:** "The one and only flipper zero Database for all your pen-testing needs"
#### Features:
- Comprehensive collection
- Regular updates
- Mixed quality (community contributions)
#### Testing Priority: **Medium**
**Best Use:** Backup source if other databases don't have specific device
---
## Secondary Databases (Specialized)
### 5. WSSFlipperZero (Weather Station Spoofer) ⭐⭐⭐⭐☆
**Repository:** https://github.com/timdigga/WSSFlipperZero
**Type:** Synthetic .sub generator
**Format:** Generated `.sub` files
**Protocols:** Acurite, LaCrosse, Oregon Scientific
#### Features:
- Generates valid weather sensor .sub files
- Uses OpenWeatherMap API for realistic data
- Known-good protocol implementations
#### Testing Priority: **High** (for synthetic testing)
**Use Case:** Generate test files with known parameters for validation
---
### 6. OWMFlipperZero (OpenWeatherMap Sync) ⭐⭐⭐☆☆
**Repository:** https://github.com/timdigga/OWMFlipperZero
**Type:** Weather .sub generator
**Format:** Generated `.sub` files
Similar to WSSFlipperZero, generates synthetic weather sensor signals.
---
### 7. Awesome Flipper Zero (Resource List) ⭐⭐⭐⭐⭐
**Repository:** https://github.com/djsime1/awesome-flipperzero
**Type:** Curated list of resources
**Content:** Links to 100+ Flipper Zero resources
#### Features:
- Curated quality resources
- Organized by category
- Regularly updated
- Links to more databases
#### Use Case:** Discovery of additional databases
---
## Additional Sources (Found via Web Search)
### 8. ADolbyB Flipper Files
**Repository:** https://github.com/ADolbyB/flipper-zero-files
**Content:** Collection from various sources
### 9. magikh0e FlipperZero_Stuff
**Repository:** https://github.com/magikh0e/FlipperZero_Stuff
**Content:** IR, Sub-GHz, remotes
### 10. honeer flipper-base
**Repository:** https://github.com/honeer/flipper-base
**Description:** Database of all things Flipper Zero
### 11. spoof_temp (Synthetic Generator)
**Repository:** https://github.com/rgerganov/spoof_temp
**Type:** Temperature sensor spoofer
**Protocols:** Oregon Scientific
---
## File Format Comparison
### .cu8 Format (RTL_433 Native)
**Description:** Complex Unsigned 8-bit I/Q samples
**Channels:** 2 (I and Q)
**Sample Rate:** Typically 250kHz
**Size:** Large (raw SDR data)
**Structure:**
```
byte[0] = I sample (0-255)
byte[1] = Q sample (0-255)
byte[2] = I sample
byte[3] = Q sample
...
```
**Pros:**
- Native RTL_433 format
- Contains full signal information
- Can be reprocessed with different decoders
**Cons:**
- Large file sizes
- Requires conversion to .sub for our use case
---
### .sub Format (Flipper Zero)
**Description:** Pulse timing array
**Variants:** KEY (decoded) or RAW (pulse data)
**Size:** Small (text file)
**Structure (RAW):**
```
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok650Async
Protocol: RAW
RAW_Data: 1024 -512 512 -1024 ...
```
**Pros:**
- Small file size
- Human-readable
- Ready to use with our parser
**Cons:**
- Only pulse timing (loses amplitude/frequency info)
- KEY format cannot be re-decoded
---
## Converting Between Formats
### .cu8 → .sub Conversion (TODO)
**Challenge:** RTL_433 test files are in `.cu8` format, we need `.sub`
**Required Steps:**
1. **Demodulate .cu8** → Extract pulse data
2. **Extract timing** → Pulse widths in microseconds
3. **Generate .sub** → Create Flipper format file
**Pseudo-code:**
```python
def convert_cu8_to_sub(cu8_file, output_sub):
"""Convert RTL_433 .cu8 to Flipper .sub"""
# 1. Load .cu8 file (I/Q samples)
iq_samples = load_cu8(cu8_file)
# 2. Demodulate (OOK/FSK)
demod_signal = demodulate_ook(iq_samples)
# 3. Extract pulse timing
pulse_data = extract_pulses(demod_signal)
# 4. Generate .sub file
sub_content = f"""Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok650Async
Protocol: RAW
RAW_Data: {' '.join(map(str, pulse_data))}
"""
with open(output_sub, 'w') as f:
f.write(sub_content)
```
**Tools Needed:**
- `scipy` for signal processing
- `numpy` for array operations
- RTL_433 source code as reference
**Priority:** Medium (would unlock 1,000+ known-good test files)
---
## Testing Strategy
### Phase 1: Quick Validation (30 minutes)
**Dataset:** FlipperZero-Subghz-DB (ready to use)
**Steps:**
1. Download FlipperZero-Subghz-DB
2. Extract weather sensor .sub files:
```bash
find FlipperZero-Subghz-DB -name "*.sub" | \
xargs grep -l "Weather\|Oregon\|Acurite\|LaCrosse" > weather.txt
```
3. Test with 10-20 files
4. Measure decode success rate
**Expected Result:** 10-30% success rate (most are proprietary protocols)
---
### Phase 2: High-Quality Testing (2 hours)
**Dataset:** UberGuidoZ Flipper (curated)
**Steps:**
1. Download UberGuidoZ database
2. Search for specific manufacturers:
```bash
find Flipper -name "*.sub" -exec grep -l "Acurite\|Oregon" {} \;
```
3. Test all matches
4. Document successes/failures
**Expected Result:** 40-60% success rate (better curation)
---
### Phase 3: Gold Standard Testing (4 hours)
**Dataset:** RTL_433 test files (requires converter)
**Steps:**
1. Clone rtl_433_tests repository
2. Create .cu8 → .sub converter
3. Convert 100 test files
4. Test all converted files
5. Validate 100% decode rate
**Expected Result:** 95-100% success rate (known good signals)
---
### Phase 4: Synthetic Testing (1 hour)
**Dataset:** WSSFlipperZero (generated)
**Steps:**
1. Clone WSSFlipperZero
2. Generate 10 synthetic weather sensor .sub files
3. Test all generated files
4. Verify decode accuracy
**Expected Result:** 80-95% success rate (synthetic signals, protocol-compliant)
---
## Download Script
Created: `scripts/download_rf_test_datasets.sh`
**Usage:**
```bash
chmod +x scripts/download_rf_test_datasets.sh
./scripts/download_rf_test_datasets.sh
```
**Downloads:**
1. rtl_433_tests (1,000+ .cu8 files)
2. FlipperZero-Subghz-DB (13,717 .sub files)
3. UberGuidoZ Flipper (~10,000 files)
4. WSSFlipperZero (synthetic generators)
5. Full_Flipper_Database (comprehensive)
6. awesome-flipperzero (resource list)
**Total Download Size:** ~1-2GB
**Total Files:** 30,000+ signals
---
## Success Metrics
### By Database
| Database | Files | Format | Expected Success |
|----------|-------|--------|------------------|
| rtl_433_tests | 1,000+ | .cu8* | 95-100% |
| UberGuidoZ | 10,000+ | .sub | 40-60% |
| FlipperZero-Subghz-DB | 13,717 | .sub | 10-30% |
| WSSFlipperZero | Generated | .sub | 80-95% |
*Requires conversion to .sub
### By Protocol Type
| Protocol Category | Expected Success |
|-------------------|------------------|
| Weather Sensors (Oregon, Acurite, LaCrosse) | 90%+ |
| TPMS (Tire Pressure) | 80%+ |
| Wireless Thermometers | 85%+ |
| Security Sensors | 70%+ |
| Doorbells | 65%+ |
| Garage Door Openers | <10% |
| Gate Remotes | <5% |
---
## Next Steps
### Immediate (High Priority)
1. **Run download script** to get datasets
```bash
./scripts/download_rf_test_datasets.sh
```
2. **Test with FlipperZero-Subghz-DB** (no conversion needed)
```bash
# Extract weather sensors
find data/rf_test_datasets/FlipperZero-Subghz-DB -name "*.sub" | \
xargs grep -l "Weather\|Oregon\|Acurite" > weather_sensors.txt
# Test first 10
head -10 weather_sensors.txt | while read file; do
echo "Testing: $file"
PYTHONPATH=. python3 scripts/test_rtl433_with_known_devices.py "$file"
done
```
3. **Measure success rate** and document results
### Short-Term (Medium Priority)
1. **Create .cu8 → .sub converter**
2. **Convert rtl_433_tests files**
3. **Achieve 95%+ decode rate** with known-good signals
### Long-Term (Low Priority)
1. **Build curated test dataset** (100 known-good .sub files)
2. **Create automated testing pipeline**
3. **Contribute findings back to rtl_433 project**
---
## References
### Web Search Queries Used
1. `flipper zero sub files database weather sensor github`
2. `rtl_433 test files samples cu8 github protocol database`
3. `"acurite" OR "oregon scientific" OR "lacrosse" weather sensor .sub file github`
### Repositories Found
- https://github.com/merbanan/rtl_433_tests
- https://github.com/Zero-Sploit/FlipperZero-Subghz-DB
- https://github.com/UberGuidoZ/Flipper
- https://github.com/DefinetlyNotAI/Full_Flipper_Database
- https://github.com/timdigga/WSSFlipperZero
- https://github.com/timdigga/OWMFlipperZero
- https://github.com/djsime1/awesome-flipperzero
- https://github.com/ADolbyB/flipper-zero-files
- https://github.com/magikh0e/FlipperZero_Stuff
- https://github.com/honeer/flipper-base
- https://github.com/rgerganov/spoof_temp
---
## Summary
We have identified **6 primary RF signal databases** with 30,000+ labeled captures:
**Best for immediate testing:**
- FlipperZero-Subghz-DB (13,717 .sub files, ready to use)
- UberGuidoZ Flipper (10,000+ files, well-organized)
**Best for high-quality validation:**
- rtl_433_tests (1,000+ files, 100% labeled, requires conversion)
**Best for synthetic testing:**
- WSSFlipperZero (generates known-good weather sensor signals)
**Next Action:** Run `./scripts/download_rf_test_datasets.sh` to download all databases and start testing!