# Signature Database Integration

## Overview

This document describes how to integrate and use signature databases from Flipper Zero, RTL_433, and community sources for device identification.

## 1. Flipper Zero Sub-GHz Database

### Source
- **Repository**: https://github.com/flipperdevices/flipperzero-firmware
- **Location**: `/assets/resources/subghz/assets/` in firmware repo
- **Format**: `.sub` files

### File Format Structure

#### Standard Key File
```
Filetype: Flipper SubGhz Key File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok270Async
Protocol: Princeton
Bit: 24
Key: 00 00 00 00 00 95 D5 D4
TE: 400
```

**Field Descriptions:**
- `Filetype`: Must be "Flipper SubGhz Key File" for protocol files
- `Version`: File format version (currently 1)
- `Frequency`: Operating frequency in Hz (e.g., 433920000 = 433.92 MHz)
- `Preset`: Modulation preset (see Preset Types below)
- `Protocol`: Protocol name (Princeton, KeeLoq, Star Line, etc.)
- `Bit`: Number of bits in the transmission
- `Key`: Hex-encoded data payload
- `TE`: Timing element in microseconds (pulse width)

#### RAW Signal File
```
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok650Async
Protocol: RAW
RAW_Data: 29262 361 -68 2635 -66 24113 -66 11 -66 11 -132
```

**RAW_Data Format:**
- Array of timing values in microseconds
- Positive values = carrier ON
- Negative values = carrier OFF
- Must alternate between positive and negative
- Up to 512 values per line

#### BinRAW File (Compressed)
```
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 315000000
Preset: FuriHalSubGhzPreset2FSKDev238Async
Protocol: BinRAW
Bit: 1572
TE: 597
Bit_RAW: 260
Data_RAW: 00 00 00 00 AA AA AA AA 0F 4A B5 55
```

**BinRAW Format:**
- `Bit`: Total bits in transmission
- `TE`: Timing element (microsecond per bit)
- `Bit_RAW`: Number of bits in compressed format
- `Data_RAW`: Bit-packed data (1=carrier, 0=gap)

### Preset Types

| Preset | Modulation | Bandwidth | Deviation | Use Case |
|--------|------------|-----------|-----------|----------|
| `FuriHalSubGhzPresetOok270Async` | OOK | 270 kHz | - | Standard garage doors, remotes |
| `FuriHalSubGhzPresetOok650Async` | OOK | 650 kHz | - | Fast protocols, doorbells |
| `FuriHalSubGhzPreset2FSKDev238Async` | 2FSK | 270 kHz | 2.38 kHz | TPMS, some sensors |
| `FuriHalSubGhzPreset2FSKDev476Async` | 2FSK | 270 kHz | 47.6 kHz | High-deviation FSK |
| `FuriHalSubGhzPresetCustom` | Custom | Varies | Varies | User-defined configs |

### Custom Presets

Custom presets allow specific CC1101 register configurations:

```
Filetype: Flipper SubGhz Key File
Version: 1
Frequency: 868350000
Preset: FuriHalSubGhzPresetCustom
Custom_preset_module: CC1101
Custom_preset_data: 02 0D 03 07 08 32 0B 06 14 00 13 00 12 00 11 32 10 17 18 18 19 18 1D 91 1C 00 1B 07 20 FB 22 11 21 B6 00 00 C0 00 00 00 00 00 00 00
Protocol: StarLine
...
```

### Common Protocols

| Protocol | Frequency | Modulation | Bit Length | Use Case |
|----------|-----------|------------|------------|----------|
| Princeton | 433.92 MHz | OOK_PWM | 24 | Generic remotes, garage doors |
| KeeLoq | 433.92 MHz | OOK | 64-66 | Car key fobs, secure remotes |
| Star Line | 433.92 MHz | OOK | Varies | Car alarm systems |
| Came | 433.92 MHz | OOK | 12 | Gate openers |
| Nice FLO | 433.92 MHz | OOK | 12-24 | Gate openers |
| Somfy Telis | 433.42 MHz | OOK | 56 | Window blinds |
| Holtek HT12X | 433.92 MHz | OOK_PWM | 12 | Generic remotes |

### Importing Flipper Signatures

#### Step 1: Clone the Repository
```bash
cd signatures/flipper
git clone --depth 1 https://github.com/flipperdevices/flipperzero-firmware.git temp
cp -r temp/assets/resources/subghz/assets/* ./
rm -rf temp
```

#### Step 2: Parse .sub Files
```python
import re
from pathlib import Path

def parse_flipper_sub(file_path):
    """Parse a Flipper Zero .sub file"""
    data = {}
    with open(file_path, 'r') as f:
        for line in f:
            line = line.strip()
            if ':' in line:
                key, value = line.split(':', 1)
                data[key.strip()] = value.strip()
    return data

# Example usage
sub_file = Path('signatures/flipper/princeton_433.sub')
parsed = parse_flipper_sub(sub_file)
print(f"Protocol: {parsed.get('Protocol')}")
print(f"Frequency: {parsed.get('Frequency')} Hz")
print(f"Key: {parsed.get('Key')}")
```

#### Step 3: Import to Database
```python
def import_flipper_signature(parsed_data, device_id):
    """Import parsed .sub file to database"""
    signature = {
        'device_id': device_id,
        'protocol': parsed_data.get('Protocol'),
        'frequency': int(parsed_data.get('Frequency', 0)),
        'modulation': parse_preset_modulation(parsed_data.get('Preset')),
        'bit_pattern': bytes.fromhex(parsed_data.get('Key', '').replace(' ', '')),
        'timing_min': int(parsed_data.get('TE', 0)) * 0.9,  # 10% tolerance
        'timing_max': int(parsed_data.get('TE', 0)) * 1.1,
        'source': 'flipper',
        'source_file': str(file_path)
    }
    # Insert into database
    db.signatures.insert(signature)
```

## 2. RTL_433 Protocol Database

### Source
- **Repository**: https://github.com/merbanan/rtl_433
- **Location**: `/src/devices/*.c` (protocol implementations)
- **Documentation**: https://triq.org/rtl_433/

### Protocol Structure

RTL_433 protocols are defined in C code with decoder specifications:

```c
static char* output_fields[] = {
    "model",
    "id",
    "channel",
    "battery_ok",
    "temperature_C",
    "humidity",
    "mic",
    NULL,
};

r_device acurite_tower = {
    .name           = "Acurite-Tower",
    .modulation     = OOK_PULSE_PWM,
    .short_width    = 220,
    .long_width     = 440,
    .reset_limit    = 900,
    .decode_fn      = &acurite_tower_decode,
    .fields         = output_fields,
};
```

### JSON Output Format

RTL_433 outputs decoded data as JSON:

```json
{
  "time": "2025-01-11 20:15:32",
  "model": "Acurite-Tower",
  "id": 12345,
  "channel": "A",
  "battery_ok": 1,
  "temperature_C": 22.5,
  "humidity": 45,
  "mic": "CRC"
}
```

### Common Fields

| Field | Type | Description |
|-------|------|-------------|
| `time` | String | Timestamp in ISO 8601 format |
| `model` | String | Manufacturer-Model identifier |
| `id` | Integer | Unique device ID |
| `channel` | String | Channel identifier (A, B, C, etc.) |
| `battery_ok` | Integer | Battery status (0=low, 1=ok) |
| `temperature_C` | Float | Temperature in Celsius |
| `humidity` | Integer | Relative humidity percentage |
| `mic` | String | Message integrity check type |

### Modulation Types

| Modulation | Description | Common Devices |
|------------|-------------|----------------|
| `OOK_PWM` | Pulse Width Modulation | Remotes, sensors |
| `OOK_PPM` | Pulse Position Modulation | Weather stations |
| `OOK_PCM` | Pulse Code Modulation | Security sensors |
| `FSK_PCM` | FSK Pulse Code | TPMS, smart meters |
| `FSK_PWM` | FSK Pulse Width | Advanced sensors |

### Extracting Protocol Definitions

#### Step 1: Extract from Source Code
```bash
cd signatures/rtl433
git clone --depth 1 https://github.com/merbanan/rtl_433.git temp
grep -r "r_device" temp/src/devices/*.c > protocols.txt
rm -rf temp
```

#### Step 2: Parse Protocol Definitions
```python
import re
import json

def parse_rtl433_protocol(protocol_string):
    """Extract protocol definition from C code"""
    pattern = r'r_device\s+(\w+)\s*=\s*{([^}]+)}'
    match = re.search(pattern, protocol_string, re.DOTALL)

    if not match:
        return None

    name = match.group(1)
    body = match.group(2)

    # Extract fields
    modulation = re.search(r'\.modulation\s*=\s*(\w+)', body)
    short_width = re.search(r'\.short_width\s*=\s*(\d+)', body)
    long_width = re.search(r'\.long_width\s*=\s*(\d+)', body)
    model_name = re.search(r'\.name\s*=\s*"([^"]+)"', body)

    return {
        'protocol_name': name,
        'model': model_name.group(1) if model_name else None,
        'modulation': modulation.group(1) if modulation else None,
        'short_width': int(short_width.group(1)) if short_width else None,
        'long_width': int(long_width.group(1)) if long_width else None
    }
```

#### Step 3: Generate Protocol Database
```python
# Read all protocol definitions
protocols = []
for c_file in Path('temp/src/devices').glob('*.c'):
    with open(c_file, 'r') as f:
        content = f.read()
        parsed = parse_rtl433_protocol(content)
        if parsed:
            protocols.append(parsed)

# Save as JSON
with open('signatures/rtl433/protocols.json', 'w') as f:
    json.dump(protocols, f, indent=2)
```

### Using rtl_433 Test Data

RTL_433 provides test signals with expected JSON output:

```bash
# Clone test repository
git clone https://github.com/merbanan/rtl_433_tests.git signatures/rtl433/tests

# Test files are organized as:
# rtl_433_tests/tests/{protocol_name}/{signal_type}/*.cu8
# rtl_433_tests/tests/{protocol_name}/{signal_type}/*.json
```

Example test data structure:
```
rtl_433_tests/tests/acurite/01/
├── g001_433.92M_250k.cu8  # Raw signal capture
└── g001_433.92M_250k.json # Expected decoded output
```

### Importing RTL_433 Signatures

```python
def import_rtl433_protocol(protocol_data, device_id):
    """Import RTL_433 protocol to database"""
    # Determine frequency from modulation
    frequency_map = {
        'OOK': 433920000,
        'FSK': 868000000  # Varies by device
    }

    modulation_type = protocol_data.get('modulation', 'OOK')
    base_freq = frequency_map.get(modulation_type[:3], 433920000)

    signature = {
        'device_id': device_id,
        'protocol': protocol_data.get('model'),
        'frequency': base_freq,
        'modulation': modulation_type,
        'timing_min': protocol_data.get('short_width'),
        'timing_max': protocol_data.get('long_width'),
        'source': 'rtl433',
        'source_file': protocol_data.get('source_file')
    }

    # Insert into rtl433_protocols table
    db.rtl433_protocols.insert({
        'protocol_name': protocol_data.get('protocol_name'),
        'model': protocol_data.get('model'),
        'modulation': modulation_type,
        'short_width': protocol_data.get('short_width'),
        'long_width': protocol_data.get('long_width'),
        'json_fields': protocol_data.get('fields', {})
    })

    # Also create general signature
    db.signatures.insert(signature)
```

## 3. Universal Radio Hacker (URH)

### Source
- **Repository**: https://github.com/jopohl/urh
- **Format**: `.urh` project files, `.complex` signal files

### Signal File Formats

#### .complex Files
Raw I/Q signal data in binary format:
- Interleaved I and Q samples
- Typically 32-bit float or 8-bit integer
- Sample rate metadata in accompanying .cfl file

#### .urh Project Files
XML-based project containing:
- Signal definitions
- Demodulation parameters
- Protocol structure
- Labeled message fields

### Extracting URH Protocols

URH projects often contain valuable protocol information in the community wiki.

```bash
# Download community-shared URH projects
cd signatures/urh
# Check URH GitHub wiki for shared project links
```

## 4. Community Signatures

### User Submission Format

When users identify a device, they can submit:

```json
{
  "device": {
    "manufacturer": "Chamberlain",
    "model": "KLIK3U-SS",
    "type": "garage_door_opener",
    "fcc_id": "K49KLIK3U"
  },
  "signature": {
    "frequency": 433920000,
    "protocol": "Security+ 2.0",
    "modulation": "OOK",
    "notes": "Rolling code, 3-button remote"
  },
  "evidence": {
    "photos": ["device_front.jpg", "device_back.jpg", "fcc_label.jpg"],
    "capture_file": "chamberlain_klik3u_001.sub"
  },
  "location": {
    "latitude": 40.7128,
    "longitude": -74.0060,
    "accuracy": 10
  }
}
```

### Verification Process

1. User submits identification with photo evidence
2. Community members vote (upvote/downvote)
3. After 5 net upvotes, identification is auto-verified
4. Verified identifications create new signature entries
5. High-reputation users can verify immediately

## 5. Signature Matching Algorithm

### Matching Strategy

```python
def match_capture_to_signatures(capture):
    """
    Match a capture against known signatures
    Returns list of (device_id, confidence) tuples
    """
    matches = []

    # Exact protocol + frequency + timing match
    exact = db.query('''
        SELECT device_id, 1.0 as confidence
        FROM signatures
        WHERE protocol = ? AND frequency = ? AND ? BETWEEN timing_min AND timing_max
    ''', [capture.protocol, capture.frequency, capture.timing_element])
    matches.extend(exact)

    # Protocol + frequency match (80% confidence)
    protocol_freq = db.query('''
        SELECT device_id, 0.8 as confidence
        FROM signatures
        WHERE protocol = ? AND frequency = ?
    ''', [capture.protocol, capture.frequency])
    matches.extend(protocol_freq)

    # Bit pattern matching (if available)
    if capture.key_data:
        pattern_matches = match_bit_pattern(capture.key_data)
        matches.extend(pattern_matches)

    # De-duplicate and sort by confidence
    unique_matches = {}
    for device_id, conf in matches:
        if device_id not in unique_matches or conf > unique_matches[device_id]:
            unique_matches[device_id] = conf

    return sorted(unique_matches.items(), key=lambda x: x[1], reverse=True)
```

### Bit Pattern Matching

```python
def match_bit_pattern(key_data, signatures):
    """
    Match key data against signature patterns with masks
    """
    matches = []

    for sig in signatures:
        if not sig.bit_mask:
            continue

        # Apply mask and compare
        masked_capture = apply_mask(key_data, sig.bit_mask)
        masked_signature = apply_mask(sig.bit_pattern, sig.bit_mask)

        if masked_capture == masked_signature:
            confidence = 0.9 * sig.weight
            matches.append((sig.device_id, confidence))

    return matches

def apply_mask(data, mask):
    """Bitwise AND operation on byte arrays"""
    return bytes(a & b for a, b in zip(data, mask))
```

## 6. Database Import Scripts

### Complete Import Pipeline

```python
#!/usr/bin/env python3
"""Import all signature databases"""

import sys
from pathlib import Path
from importers import flipper, rtl433, urh

def main():
    print("Starting signature database import...")

    # Import Flipper Zero signatures
    print("\n[1/3] Importing Flipper Zero .sub files...")
    flipper_dir = Path('signatures/flipper')
    flipper_count = flipper.import_all(flipper_dir)
    print(f"  Imported {flipper_count} Flipper signatures")

    # Import RTL_433 protocols
    print("\n[2/3] Importing RTL_433 protocols...")
    rtl433_file = Path('signatures/rtl433/protocols.json')
    rtl433_count = rtl433.import_protocols(rtl433_file)
    print(f"  Imported {rtl433_count} RTL_433 protocols")

    # Import URH community signals
    print("\n[3/3] Importing URH signals...")
    urh_dir = Path('signatures/urh')
    urh_count = urh.import_all(urh_dir)
    print(f"  Imported {urh_count} URH signatures")

    print(f"\nTotal signatures imported: {flipper_count + rtl433_count + urh_count}")

if __name__ == '__main__':
    main()
```

## 7. Signature Database Maintenance

### Regular Updates

```bash
#!/bin/bash
# scripts/update_signatures.sh

cd signatures/flipper
git pull

cd ../rtl433
git pull

# Re-import updated signatures
python3 scripts/import_signatures.py --update
```

### Quality Metrics

Track signature effectiveness:

```sql
-- Signature match success rate
SELECT
    s.id,
    d.manufacturer,
    d.model,
    COUNT(c.id) as total_matches,
    AVG(c.match_confidence) as avg_confidence
FROM signatures s
JOIN devices d ON s.device_id = d.id
LEFT JOIN captures c ON c.device_id = d.id AND c.match_method = 'auto'
GROUP BY s.id, d.manufacturer, d.model
ORDER BY total_matches DESC;
```