# RF Device Identification - Benchmark Results

**Date**: 1771170119.1678076
**Total Tests**: 12

## Overall Metrics

### Top-K Accuracy

- **Top-1**: 33.3%
- **Top-3**: 33.3%
- **Top-5**: 33.3%

### Confidence Distribution

- **High (>80%)**: 7 (58.3%)
- **Medium (50-80%)**: 4 (33.3%)
- **Low (<50%)**: 1 (8.3%)

### Performance

- **Avg Parse Time**: 0.40 ms
- **Avg Match Time**: 156.29 ms
- **Total**: 156.69 ms

## Per-Protocol Results

| Protocol | Tests | Top-1 Acc | Avg Confidence |
|----------|-------|-----------|----------------|
| Acurite 609TXC | 1 | 0.0% | 86.4% |
| Nexus Temperature-Humidity | 1 | 0.0% | 86.2% |
| Princeton | 2 | 0.0% | 69.3% |
| PT2262 | 1 | 0.0% | 87.5% |
| Toyota TPMS | 1 | 0.0% | 67.7% |
| Honeywell Security | 1 | 0.0% | 0.0% |
| Generic Doorbell | 1 | 0.0% | 84.8% |
| LaCrosse TX141-BV2 | 2 | 100.0% | 97.4% |
| Oregon Scientific v2.1 | 1 | 100.0% | 90.9% |
| Schrader TPMS | 1 | 100.0% | 65.7% |

## Detailed Results

### ✓ lacrosse_tx141-bv2_synthetic.sub

- **Expected**: LaCrosse TX141-BV2
- **Got**: LaCrosse TX141TH-Bv2 (confidence: 98.8%)
- **Rank**: 1

**Top 5 Matches**:
1. LaCrosse TX141TH-Bv2 (98.8%)
2. ELV EM 1000 (86.3%)
3. Funkbus / Instafunk (Berker, Gira, Jung) (86.3%)
4. Wireless M-Bus, Mode T, 32.768kbps (-f 868.3M -s 1000k) (86.3%)
5. Wireless M-Bus, Mode S, 32.768kbps (-f 868.3M -s 1000k) (86.3%)

### ✗ acurite_609txc_synthetic.sub

- **Expected**: Acurite 609TXC
- **Got**: Clipsal CMR113 Cent-a-meter power meter (confidence: 86.4%)
- **Rank**: 118

**Top 5 Matches**:
1. Clipsal CMR113 Cent-a-meter power meter (86.4%)
2. Norgo NGE101 (86.2%)
3. Holman Industries iWeather WS5029 weather station (older PWM) (86.1%)
4. ELV EM 1000 (85.2%)
5. Funkbus / Instafunk (Berker, Gira, Jung) (85.2%)

### ✓ oregon_scientific_v2.1_synthetic.sub

- **Expected**: Oregon Scientific v2.1
- **Got**: Oregon Scientific v2.1 (confidence: 90.9%)
- **Rank**: 1

**Top 5 Matches**:
1. Oregon Scientific v2.1 (90.9%)
2. Oregon Scientific v3.0 (90.0%)
3. Oregon Scientific Weather Sensor (88.4%)
4. LaCrosse TX141TH-Bv2 (87.5%)
5. Clipsal CMR113 Cent-a-meter power meter (76.4%)

### ✗ nexus_temperature-humidity_synthetic.sub

- **Expected**: Nexus Temperature-Humidity
- **Got**: Holman Industries iWeather WS5029 weather station (older PWM) (confidence: 86.2%)
- **Rank**: 63

**Top 5 Matches**:
1. Holman Industries iWeather WS5029 weather station (older PWM) (86.2%)
2. Norgo NGE101 (86.1%)
3. ELV EM 1000 (85.9%)
4. Funkbus / Instafunk (Berker, Gira, Jung) (85.9%)
5. Wireless M-Bus, Mode T, 32.768kbps (-f 868.3M -s 1000k) (85.9%)

### ✗ princeton_synthetic.sub

- **Expected**: Princeton
- **Got**: SimpliSafe Home Security System (May require disabling automatic gain for KeyPad decodes) (confidence: 79.4%)
- **Rank**: Not Found

**Top 5 Matches**:
1. SimpliSafe Home Security System (May require disabling automatic gain for KeyPad decodes) (79.4%)
2. Cardin S466-TX2 (57.5%)
3. Akhan 100F14 remote keyless entry (42.9%)
4. Chamberlain/LiftMaster (37.9%)

### ✗ pt2262_synthetic.sub

- **Expected**: PT2262
- **Got**: Princeton (confidence: 87.5%)
- **Rank**: 7

**Top 5 Matches**:
1. Princeton (87.5%)
2. Waveman Switch Transmitter (86.2%)
3. Quhwa (85.9%)
4. ELV WS 2000 (85.4%)
5. Intertechno 433 (84.0%)

### ✓ schrader_tpms_synthetic.sub

- **Expected**: Schrader TPMS
- **Got**: Schrader TPMS SMD3MA4 (Subaru) 3039 (Infiniti, Nissan, Renault) (confidence: 65.7%)
- **Rank**: 1

**Top 5 Matches**:
1. Schrader TPMS SMD3MA4 (Subaru) 3039 (Infiniti, Nissan, Renault) (65.7%)
2. Nissan TPMS (65.7%)
3. AVE TPMS (53.8%)
4. PMV-107J (Toyota) TPMS (53.8%)
5. TyreGuard 400 TPMS (53.8%)

### ✗ toyota_tpms_synthetic.sub

- **Expected**: Toyota TPMS
- **Got**: Schrader TPMS SMD3MA4 (Subaru) 3039 (Infiniti, Nissan, Renault) (confidence: 67.7%)
- **Rank**: Not Found

**Top 5 Matches**:
1. Schrader TPMS SMD3MA4 (Subaru) 3039 (Infiniti, Nissan, Renault) (67.7%)
2. Nissan TPMS (67.7%)
3. AVE TPMS (55.7%)
4. PMV-107J (Toyota) TPMS (55.7%)
5. TyreGuard 400 TPMS (55.7%)

### ✗ honeywell_security_synthetic.sub

- **Expected**: Honeywell Security
- **Got**: None (confidence: 0.0%)
- **Rank**: Not Found

### ✗ generic_doorbell_synthetic.sub

- **Expected**: Generic Doorbell
- **Got**: Brennenstuhl RCS 2044 (confidence: 84.8%)
- **Rank**: Not Found

**Top 5 Matches**:
1. Brennenstuhl RCS 2044 (84.8%)
2. PT2262 (84.4%)
3. Waveman Switch Transmitter (81.3%)
4. Silvercrest Remote Control (81.2%)
5. Quhwa (81.1%)

### ✓ lacrosse_tx141-bv2_noisy_synthetic.sub

- **Expected**: LaCrosse TX141-BV2
- **Got**: LaCrosse TX141TH-Bv2 (confidence: 96.0%)
- **Rank**: 1

**Top 5 Matches**:
1. LaCrosse TX141TH-Bv2 (96.0%)
2. Opus/Imagintronix XT300 Soil Moisture (86.4%)
3. DSC Security Contact (WS4945) (86.0%)
4. Acurite 986 Refrigerator / Freezer Thermometer (85.0%)
5. Digitech XC-0324 / AmbientWeather FT005TH temp/hum sensor (85.0%)

### ✗ princeton_noisy_synthetic.sub

- **Expected**: Princeton
- **Got**: Cardin S466-TX2 (confidence: 59.2%)
- **Rank**: Not Found

**Top 5 Matches**:
1. Cardin S466-TX2 (59.2%)
2. SimpliSafe Home Security System (May require disabling automatic gain for KeyPad decodes) (44.8%)
3. Akhan 100F14 remote keyless entry (40.3%)
4. Chamberlain/LiftMaster (35.0%)