giglez/docs/CLEANUP_AND_OPTIMIZATION_PLAN.md

# GigLez Cleanup & Optimization Plan

**Date:** 2026-01-14
**Purpose:** Identify bloat, unused code, and plan optimizations + JavaScript implementation

---

## Part 1: Code Bloat Analysis

### A. Duplicate/Redundant Matcher Implementations

#### 🔴 HIGH PRIORITY - Consolidate Matchers

**Current State:**
```
src/matcher/
├── engine.py                    # Base MatchStrategy interface (3.4 KB)
├── strategies.py                # NEW unified strategy system (610 lines) ✅
├── strategies_orm.py            # OLD ORM-based strategies (356 lines) ❌ REDUNDANT
├── simple_matcher.py            # Standalone matcher (351 lines) ❌ REDUNDANT
├── rtl433_matcher.py            # Standalone RTL_433 (352 lines) ❌ REDUNDANT
├── rtl433_decoder.py            # RTL_433 decoder (385 lines) ✅ USED
├── pattern_decoder.py           # Pattern decoder (461 lines) ✅ USED
└── protocol_database.py         # Protocol DB (435 lines) ✅ USED
```

**Problem:**
- **3 different matcher implementations** doing similar things
- `strategies.py` is the NEW unified system (includes all strategies)
- `strategies_orm.py`, `simple_matcher.py`, `rtl433_matcher.py` are OLD and redundant

**Evidence:**
```python
# main_simple.py uses:
from src.matcher.simple_matcher import get_matcher  # OLD
from src.matcher.rtl433_decoder import get_decoder  # OK

# strategies.py contains:
class ExactMatcher(MatchStrategy)           # Replaces simple_matcher
class FrequencyMatcher(MatchStrategy)       # Replaces simple_matcher
class BitPatternMatcher(MatchStrategy)      # Replaces simple_matcher
class TimingMatcher(MatchStrategy)          # Replaces simple_matcher
class RTL433DecoderStrategy(MatchStrategy)  # Wraps rtl433_decoder
class PatternBasedStrategy(MatchStrategy)   # NEW - pattern_decoder
```

**Action Plan:**
1. ✅ **Keep**: `strategies.py` (unified system)
2. ✅ **Keep**: `rtl433_decoder.py` (used by RTL433DecoderStrategy)
3. ✅ **Keep**: `pattern_decoder.py` (used by PatternBasedStrategy)
4. ✅ **Keep**: `protocol_database.py` (used by pattern_decoder)
5. ✅ **Keep**: `engine.py` (base interfaces)
6. ❌ **DELETE**: `strategies_orm.py` (old, unused)
7. ❌ **DELETE**: `simple_matcher.py` (replaced by strategies.py)
8. ❌ **DELETE**: `rtl433_matcher.py` (replaced by strategies.py)

**Impact:**
- Remove ~1,059 lines of redundant code
- Eliminate maintenance burden of 3 parallel implementations
- Simplify API by using only `strategies.py`

---

### B. Dual API Implementations

#### 🟡 MEDIUM PRIORITY - Consolidate APIs

**Current State:**
```
src/api/
├── main.py                # Full ORM-based API (280 lines)
└── main_simple.py         # Simplified API (536 lines) ✅ CURRENTLY USED
```

**Problem:**
- Two FastAPI implementations
- `main_simple.py` is actively developed and used
- `main.py` uses old ORM approach (SQLAlchemy)
- Confusion about which is "production"

**Recommendation:**
1. ✅ **Keep**: `main_simple.py` (rename to `main.py` or `app.py`)
2. ❌ **Archive**: Current `main.py` → `main_orm_legacy.py` or delete

**Reason:**
- `main_simple.py` has RTL_433 integration
- `main_simple.py` uses modern psycopg2 approach
- Less abstraction = easier debugging

---

### C. Unused/Redundant Scripts

#### 🟢 LOW PRIORITY - Archive Old Scripts

**Current State:** 19 Python scripts in `scripts/`

**Categories:**

**1. Active/Essential (Keep):**
- ✅ `test_pattern_decoder.py` - New pattern decoder tests
- ✅ `test_real_weather_sensors.py` - RTL_433 validation
- ✅ `test_rtl433_with_known_devices.py` - RTL_433 tests
- ✅ `parse_rtl433_devices.py` - RTL_433 device catalog builder
- ✅ `download_rf_test_datasets.sh` - Test data fetcher

**2. Utility/Maintenance (Keep):**
- ✅ `cleanup_database.py` - Database maintenance
- ✅ `import_wardriving_data.py` - CSV import for wardriving
- ✅ `create_test_dataset.py` - Test data generator

**3. One-Time/Experimental (Archive):**
- 🔄 `analyze_flipper_signatures.py` - One-time analysis
- 🔄 `analyze_tembed_files.py` - One-time analysis
- 🔄 `identify_tembed_devices.py` - One-time analysis
- 🔄 `import_flipper_sqlite.py` - One-time import
- 🔄 `import_tembed_signatures.py` - One-time import
- 🔄 `match_tembed_with_db.py` - Exploratory
- 🔄 `match_with_flipper_db.py` - Exploratory
- 🔄 `rematch_captures.py` - One-time rematch
- 🔄 `test_gps_extraction.py` - Early prototype
- 🔄 `test_tembed_matching.py` - Early prototype
- 🔄 `test_wardriving_import.py` - Import test

**Action Plan:**
Create `scripts/archive/` directory and move 11 one-time scripts there.

**Impact:**
- Cleaner `scripts/` directory (8 files vs 19)
- Keep history but reduce clutter
- Easy to restore if needed

---

### D. Unused Parser Modules

#### 🟢 LOW PRIORITY - Review Parsers

**Current State:**
```
src/parser/
├── sub_parser.py              # Main .sub parser ✅ USED
├── rtl433_converter.py        # RAW → RTL_433 converter ✅ USED
├── metadata.py                # SignalMetadata class ✅ USED
├── raw_parser.py              # RAW signal analyzer (326 lines) ?
├── gps_extractor.py           # GPS from filenames (241 lines) ?
└── wardriving_importer.py     # CSV wardriving import (333 lines) ?
```

**Questions:**
1. Is `raw_parser.py` still used? (Seems superseded by pattern_decoder.py)
2. Is `gps_extractor.py` actively used?
3. Is `wardriving_importer.py` production-ready?

**Recommendation:**
- Check imports/usage in API
- If unused, move to `deprecated/` folder
- Document if they're experimental

---

### E. Database Connection Complexity

#### 🟡 MEDIUM PRIORITY - Simplify Database Layer

**Current State:**
```
Database layer has 3 approaches:
1. config/database.py          # Psycopg2 connection pooling
2. src/database/connection.py  # SQLAlchemy ORM
3. src/database/models.py      # SQLAlchemy models (564 lines)
```

**Problem:**
- `main_simple.py` uses psycopg2 approach (simpler)
- `main.py` uses SQLAlchemy ORM approach (complex)
- `strategies_orm.py` uses SQLAlchemy models
- Dual maintenance burden

**Recommendation:**
Since we're keeping `main_simple.py`:
1. ✅ **Keep**: `config/database.py` (psycopg2 pooling)
2. 🔄 **Archive**: `src/database/connection.py` (if not used elsewhere)
3. 🔄 **Archive**: `src/database/models.py` (if only used by old main.py)

---

## Part 2: Optimization Opportunities

### A. Pattern Decoder Enhancements

#### 🔵 ENHANCEMENT - Improve Decode Rate

**Current Performance:**
- 44.4% decode rate (4/9 files)
- Confidence range: 33-76%

**Optimization Ideas:**

**1. Lower Confidence Thresholds (Easy - 30 min)**
```python
# Current
proto.min_confidence = 0.6  # Too strict?

# Proposed
proto.min_confidence = 0.4  # Allow more matches
```

**2. Add More Protocols (Medium - 2 hours)**
- Currently: 18 protocols
- Target: 50+ protocols
- Source: Flipper Zero firmware protocol definitions

**3. Improve Timing Tolerance (Easy - 1 hour)**
```python
# Current
timing_tolerance = 0.2  # ±20%

# Proposed: Adaptive tolerance based on pulse count
if pulse_count < 150:
    timing_tolerance = 0.3  # ±30% for short captures
else:
    timing_tolerance = 0.2  # ±20% for longer captures
```

**4. Add Preamble Pattern Matching (Medium - 2 hours)**
```python
# Currently not fully utilized
if proto.preamble_pattern and proto.preamble_pattern in bit_pattern:
    pattern_confidence = 1.0
```

Enhance to use fuzzy pattern matching (Levenshtein distance).

---

### B. API Integration of Pattern Decoder

#### 🔴 CRITICAL - Pattern Decoder Not Integrated!

**Problem:**
Pattern decoder exists in `strategies.py` but is **NOT** used by API!

**Current API Flow:**
```python
# main_simple.py
from src.matcher.simple_matcher import get_matcher  # OLD
from src.matcher.rtl433_decoder import get_decoder  # OK

# Upload handler uses:
matcher = get_matcher()  # Returns OLD simple_matcher
matches = matcher.match_device(metadata)  # DOESN'T use strategies.py!

decoder = get_rtl433_decoder()  # Separate call
rtl433_devices = decoder.decode(metadata)
```

**What's Missing:**
The new unified `strategies.py` system with `PatternBasedStrategy` is **NOT CONNECTED** to the API!

**Fix Required:**
```python
# NEW approach (use strategies.py):
from src.matcher.engine import MatchEngine
from src.matcher.strategies import (
    ExactMatcher,
    FrequencyMatcher,
    BitPatternMatcher,
    TimingMatcher,
    RTL433DecoderStrategy,
    PatternBasedStrategy  # NEW!
)

# Initialize engine
engine = MatchEngine()
engine.add_strategy(ExactMatcher())
engine.add_strategy(FrequencyMatcher())
engine.add_strategy(BitPatternMatcher())
engine.add_strategy(TimingMatcher())
engine.add_strategy(RTL433DecoderStrategy())  # Uses rtl433_decoder
engine.add_strategy(PatternBasedStrategy())   # Uses pattern_decoder

# In upload handler:
matches = engine.match(metadata, db)  # Returns all matches from all strategies
```

**Impact:**
- Pattern decoder will finally be used in production!
- Unified matching pipeline
- Easier to add new strategies

---

### C. Frontend JavaScript Implementation

#### 🔵 NEW FEATURE - Client-Side Pattern Decoder

**Why JavaScript?**
1. Decode signals client-side before upload (privacy)
2. Real-time feedback during capture
3. Reduce server load
4. Works offline (PWA capability)

**Use Cases:**
1. **Web Interface**: Decode .sub files in browser before upload
2. **Mobile App**: Real-time decoding during T-Embed capture
3. **Offline Mode**: Decode signals without server

---

## Part 3: JavaScript Implementation Plan

### Architecture: Port Pattern Decoder to JavaScript

#### Target: Node.js + Browser-Compatible Module

**File Structure:**
```
src/matcher/js/
├── protocol-database.js        # Protocol signatures (ES6 module)
├── pattern-decoder.js          # Core decoder logic
├── pulse-analyzer.js           # K-means clustering for pulse widths
├── fingerprint.js              # Statistical fingerprinting
└── index.js                    # Main export
```

---

### JavaScript Implementation: Core Components

#### 1. Protocol Database (protocol-database.js)

```javascript
/**
 * RF Protocol Signature Database
 * Port of protocol_database.py
 */

export class ProtocolSignature {
  constructor({
    name,
    category,
    manufacturer = null,
    frequency = 433920000,
    shortPulseUs = 500,
    longPulseUs = 1000,
    timingTolerance = 0.2,
    preamblePattern = null,
    syncPattern = null,
    minBits = 24,
    maxBits = 64,
    typicalPulseCount = 100
  }) {
    this.name = name;
    this.category = category;
    this.manufacturer = manufacturer;
    this.frequency = frequency;
    this.shortPulseUs = shortPulseUs;
    this.longPulseUs = longPulseUs;
    this.timingTolerance = timingTolerance;
    this.preamblePattern = preamblePattern;
    this.syncPattern = syncPattern;
    this.minBits = minBits;
    this.maxBits = maxBits;
    this.typicalPulseCount = typicalPulseCount;
  }

  matchesTiming(shortUs, longUs) {
    const shortMin = this.shortPulseUs * (1 - this.timingTolerance);
    const shortMax = this.shortPulseUs * (1 + this.timingTolerance);
    const longMin = this.longPulseUs * (1 - this.timingTolerance);
    const longMax = this.longPulseUs * (1 + this.timingTolerance);

    return (shortMin <= shortUs && shortUs <= shortMax) &&
           (longMin <= longUs && longUs <= longMax);
  }

  matchesFrequency(freq) {
    const tolerance = 100000; // ±100 kHz
    return Math.abs(freq - this.frequency) <= tolerance;
  }
}

// Weather Sensors
export const WEATHER_SENSORS = [
  new ProtocolSignature({
    name: 'Oregon Scientific v2.1',
    category: 'Weather Sensor',
    manufacturer: 'Oregon Scientific',
    shortPulseUs: 488,
    longPulseUs: 976,
    preamblePattern: '10101010'.repeat(4),
    minBits: 64,
    maxBits: 128,
    typicalPulseCount: 200
  }),
  new ProtocolSignature({
    name: 'Acurite Tower Sensor',
    category: 'Weather Sensor',
    manufacturer: 'Acurite',
    shortPulseUs: 220,
    longPulseUs: 440,
    minBits: 56,
    maxBits: 64,
    typicalPulseCount: 130
  }),
  new ProtocolSignature({
    name: 'LaCrosse TX141TH-Bv2',
    category: 'Weather Sensor',
    manufacturer: 'LaCrosse',
    shortPulseUs: 500,
    longPulseUs: 1000,
    minBits: 40,
    maxBits: 48,
    typicalPulseCount: 100
  }),
  // ... add all 18 protocols
];

export const ALL_PROTOCOLS = [
  ...WEATHER_SENSORS,
  // ...GARAGE_DOOR_OPENERS,
  // ...DOORBELLS,
  // ...TIRE_PRESSURE,
  // ...SECURITY_SENSORS,
  // ...REMOTE_CONTROLS
];

export class ProtocolDatabase {
  constructor() {
    this.protocols = ALL_PROTOCOLS;
    this._indexProtocols();
  }

  _indexProtocols() {
    this.byCategory = {};
    this.byFrequency = {};

    for (const proto of this.protocols) {
      // Index by category
      if (!this.byCategory[proto.category]) {
        this.byCategory[proto.category] = [];
      }
      this.byCategory[proto.category].push(proto);

      // Index by frequency (MHz)
      const freqMHz = Math.round(proto.frequency / 1_000_000);
      if (!this.byFrequency[freqMHz]) {
        this.byFrequency[freqMHz] = [];
      }
      this.byFrequency[freqMHz].push(proto);
    }
  }

  findByTiming(shortUs, longUs, frequency = null) {
    const matches = [];
    let candidates = this.protocols;

    if (frequency) {
      const freqMHz = Math.round(frequency / 1_000_000);
      candidates = this.byFrequency[freqMHz] || this.protocols;
    }

    for (const proto of candidates) {
      if (proto.matchesTiming(shortUs, longUs)) {
        if (!frequency || proto.matchesFrequency(frequency)) {
          matches.push(proto);
        }
      }
    }

    return matches;
  }

  findByFrequency(frequency) {
    return this.protocols.filter(p => p.matchesFrequency(frequency));
  }
}
```

---

#### 2. Pulse Analyzer (pulse-analyzer.js)

```javascript
/**
 * Pulse Width Analysis using K-means clustering
 * Port of PatternDecoder._identify_pulse_widths()
 */

/**
 * Simple K-means clustering for 1D data
 */
export function kMeans(data, k = 2, maxIterations = 100) {
  if (data.length < k) {
    return data.map(v => [v]);
  }

  // Initialize centroids (use min and max)
  const sorted = [...data].sort((a, b) => a - b);
  let centroids = [sorted[0], sorted[sorted.length - 1]];

  for (let iter = 0; iter < maxIterations; iter++) {
    // Assign points to nearest centroid
    const clusters = Array.from({ length: k }, () => []);

    for (const point of data) {
      const distances = centroids.map(c => Math.abs(point - c));
      const nearestIdx = distances.indexOf(Math.min(...distances));
      clusters[nearestIdx].push(point);
    }

    // Update centroids
    const newCentroids = clusters.map(cluster => {
      if (cluster.length === 0) return centroids[0];
      return cluster.reduce((a, b) => a + b, 0) / cluster.length;
    });

    // Check convergence
    const converged = centroids.every((c, i) =>
      Math.abs(c - newCentroids[i]) < 1
    );

    centroids = newCentroids;

    if (converged) break;
  }

  return centroids.sort((a, b) => a - b);
}

/**
 * Identify SHORT and LONG pulse widths from pulse train
 */
export function identifyPulseWidths(pulses) {
  // Separate HIGH pulses (positive) and LOW gaps (negative)
  const highPulses = pulses.filter(p => p > 0).map(Math.abs);
  const lowPulses = pulses.filter(p => p < 0).map(Math.abs);

  // Cluster into SHORT/LONG
  const [shortPulse, longPulse] = highPulses.length >= 2
    ? kMeans(highPulses, 2)
    : [highPulses[0] || 0, highPulses[0] || 0];

  const [shortGap, longGap] = lowPulses.length >= 2
    ? kMeans(lowPulses, 2)
    : [lowPulses[0] || 0, lowPulses[0] || 0];

  return { shortPulse, longPulse, shortGap, longGap };
}

/**
 * Decode pulse train to binary string (PWM encoding)
 */
export function decodeToBits(pulses, shortPulse, longPulse) {
  const threshold = (shortPulse + longPulse) / 2;
  const bits = [];

  for (const pulse of pulses) {
    if (pulse > 0) {  // Only decode HIGH pulses
      bits.push(Math.abs(pulse) < threshold ? '0' : '1');
    }
  }

  return bits.join('');
}
```

---

#### 3. Statistical Fingerprinting (fingerprint.js)

```javascript
/**
 * Extract statistical fingerprint from pulse data
 */

export function extractFingerprint(pulses) {
  const highPulses = pulses.filter(p => p > 0);
  const lowPulses = pulses.filter(p => p < 0).map(Math.abs);

  // Calculate statistics
  const meanPulse = mean(highPulses);
  const stdPulse = std(highPulses);
  const meanGap = mean(lowPulses);
  const stdGap = std(lowPulses);

  const totalHigh = sum(highPulses);
  const totalLow = sum(lowPulses);
  const totalTime = totalHigh + totalLow;

  const pulseGapRatio = meanGap > 0 ? meanPulse / meanGap : 0;
  const dutyCycle = totalTime > 0 ? totalHigh / totalTime : 0;

  return {
    meanPulseWidth: meanPulse,
    stdPulseWidth: stdPulse,
    meanGapWidth: meanGap,
    stdGapWidth: stdGap,
    pulseGapRatio,
    dutyCycle,
    pulseCount: pulses.length,
    minPulse: Math.min(...highPulses),
    maxPulse: Math.max(...highPulses),
    minGap: Math.min(...lowPulses),
    maxGap: Math.max(...lowPulses)
  };
}

// Helper functions
function mean(arr) {
  return arr.length > 0 ? sum(arr) / arr.length : 0;
}

function sum(arr) {
  return arr.reduce((a, b) => a + b, 0);
}

function std(arr) {
  const m = mean(arr);
  const variance = arr.reduce((acc, val) => acc + Math.pow(val - m, 2), 0) / arr.length;
  return Math.sqrt(variance);
}
```

---

#### 4. Pattern Decoder (pattern-decoder.js)

```javascript
/**
 * Main Pattern Decoder
 * Port of PatternDecoder class
 */

import { ProtocolDatabase } from './protocol-database.js';
import { identifyPulseWidths, decodeToBits } from './pulse-analyzer.js';
import { extractFingerprint } from './fingerprint.js';

export class DeviceMatch {
  constructor(protocol, confidence, matchMethod, details) {
    this.protocol = protocol;
    this.confidence = confidence;
    this.matchMethod = matchMethod;
    this.details = details;
  }

  get name() { return this.protocol.name; }
  get manufacturer() { return this.protocol.manufacturer; }
  get category() { return this.protocol.category; }
}

export class PatternDecoder {
  constructor() {
    this.protocolDb = new ProtocolDatabase();
  }

  /**
   * Decode device from signal metadata
   * @param {Object} metadata - { frequency, raw_data: [] }
   * @returns {DeviceMatch[]}
   */
  decode(metadata) {
    if (!metadata.raw_data || metadata.raw_data.length === 0) {
      return [];
    }

    const pulses = metadata.raw_data;
    const frequency = metadata.frequency;

    const matches = [];

    // Strategy 1: Timing Pattern Analysis
    matches.push(...this._decodeTimingPatterns(pulses, frequency));

    // Strategy 2: Statistical Fingerprinting
    const fingerprint = extractFingerprint(pulses);
    matches.push(...this._matchFingerprint(fingerprint, frequency));

    // Deduplicate and rank
    return this._rankMatches(matches);
  }

  _decodeTimingPatterns(pulses, frequency) {
    const matches = [];

    // Identify pulse widths
    const { shortPulse, longPulse } = identifyPulseWidths(pulses);

    if (shortPulse === 0 || longPulse === 0) {
      return [];
    }

    // Decode to binary
    const bitPattern = decodeToBits(pulses, shortPulse, longPulse);

    // Find matching protocols
    const protocolMatches = this.protocolDb.findByTiming(
      shortPulse,
      longPulse,
      frequency
    );

    for (const proto of protocolMatches) {
      // Calculate confidence
      const timingError = Math.abs(proto.shortPulseUs - shortPulse) / proto.shortPulseUs;
      const timingConfidence = Math.max(0, 1.0 - timingError);

      const bitCount = bitPattern.length;
      const bitCountMatch = (proto.minBits <= bitCount && bitCount <= proto.maxBits);
      const bitConfidence = bitCountMatch ? 1.0 : 0.5;

      let patternConfidence = 0.7;
      if (proto.preamblePattern && bitPattern.includes(proto.preamblePattern)) {
        patternConfidence = 1.0;
      } else if (proto.syncPattern && bitPattern.includes(proto.syncPattern)) {
        patternConfidence = 0.9;
      }

      const overallConfidence = (
        timingConfidence * 0.4 +
        bitConfidence * 0.3 +
        patternConfidence * 0.3
      );

      if (overallConfidence >= 0.6) {
        matches.push(new DeviceMatch(
          proto,
          overallConfidence,
          'timing_pattern',
          {
            shortPulseUs: shortPulse,
            longPulseUs: longPulse,
            bitCount,
            bitPattern: bitPattern.substring(0, 64),
            timingError: `${(timingError * 100).toFixed(2)}%`
          }
        ));
      }
    }

    return matches;
  }

  _matchFingerprint(fingerprint, frequency) {
    const matches = [];
    const candidates = this.protocolDb.findByFrequency(frequency);

    for (const proto of candidates) {
      const expectedMeanPulse = (proto.shortPulseUs + proto.longPulseUs) / 2;
      const pulseError = Math.abs(expectedMeanPulse - fingerprint.meanPulseWidth) / expectedMeanPulse;

      const expectedCount = proto.typicalPulseCount;
      const countError = Math.abs(expectedCount - fingerprint.pulseCount) / expectedCount;

      const pulseSimilarity = Math.max(0, 1.0 - pulseError);
      const countSimilarity = Math.max(0, 1.0 - countError);

      const overallConfidence = (pulseSimilarity * 0.6 + countSimilarity * 0.4) * 0.8;

      if (overallConfidence >= 0.4) {
        matches.push(new DeviceMatch(
          proto,
          overallConfidence,
          'fingerprint',
          {
            meanPulseUs: fingerprint.meanPulseWidth.toFixed(1),
            pulseCount: fingerprint.pulseCount,
            dutyCycle: `${(fingerprint.dutyCycle * 100).toFixed(2)}%`,
            pulseError: `${(pulseError * 100).toFixed(2)}%`,
            countError: `${(countError * 100).toFixed(2)}%`
          }
        ));
      }
    }

    return matches;
  }

  _rankMatches(matches) {
    // Group by protocol name
    const byProtocol = {};

    for (const match of matches) {
      const name = match.protocol.name;
      if (!byProtocol[name]) {
        byProtocol[name] = [];
      }
      byProtocol[name].push(match);
    }

    // Keep best match per protocol
    const bestMatches = [];
    for (const protoMatches of Object.values(byProtocol)) {
      const best = protoMatches.reduce((a, b) =>
        a.confidence > b.confidence ? a : b
      );
      bestMatches.push(best);
    }

    // Sort by confidence
    return bestMatches.sort((a, b) => b.confidence - a.confidence);
  }
}
```

---

#### 5. Main Export (index.js)

```javascript
/**
 * GigLez Pattern Decoder - JavaScript Implementation
 * Browser and Node.js compatible
 */

export { PatternDecoder, DeviceMatch } from './pattern-decoder.js';
export { ProtocolDatabase, ProtocolSignature } from './protocol-database.js';
export { identifyPulseWidths, decodeToBits, kMeans } from './pulse-analyzer.js';
export { extractFingerprint } from './fingerprint.js';

// Default export for convenience
import { PatternDecoder } from './pattern-decoder.js';
export default PatternDecoder;
```

---

### JavaScript Integration: Usage Examples

#### Example 1: Node.js (Backend)

```javascript
import PatternDecoder from './src/matcher/js/index.js';

// Parse .sub file (assume you have this)
const metadata = {
  frequency: 433920000,
  raw_data: [788, -1004, 1194, -68, 3094, -162, 786, -66, 164, -200, ...]
};

// Decode
const decoder = new PatternDecoder();
const matches = decoder.decode(metadata);

console.log(`Found ${matches.length} matches:`);
for (const match of matches) {
  console.log(`  ${match.name} (${match.manufacturer}) - ${(match.confidence * 100).toFixed(1)}%`);
}
```

#### Example 2: Browser (Frontend)

```html
<!DOCTYPE html>
<html>
<head>
  <title>GigLez Pattern Decoder</title>
</head>
<body>
  <h1>Decode .sub File</h1>
  <input type="file" id="fileInput" accept=".sub">
  <div id="results"></div>

  <script type="module">
    import PatternDecoder from './src/matcher/js/index.js';

    const decoder = new PatternDecoder();

    document.getElementById('fileInput').addEventListener('change', async (e) => {
      const file = e.target.files[0];
      const text = await file.text();

      // Parse .sub file (simple parser)
      const metadata = parseSubFile(text);

      // Decode
      const matches = decoder.decode(metadata);

      // Display results
      const resultsDiv = document.getElementById('results');
      resultsDiv.innerHTML = `
        <h2>Found ${matches.length} matches:</h2>
        <ul>
          ${matches.map(m => `
            <li>
              <strong>${m.name}</strong> (${m.manufacturer || 'Unknown'})
              <br>Confidence: ${(m.confidence * 100).toFixed(1)}%
              <br>Method: ${m.matchMethod}
            </li>
          `).join('')}
        </ul>
      `;
    });

    function parseSubFile(text) {
      // Simple .sub parser (extract frequency and RAW_Data)
      const frequencyMatch = text.match(/Frequency: (\d+)/);
      const rawDataMatch = text.match(/RAW_Data: ([\d\s-]+)/);

      if (!frequencyMatch || !rawDataMatch) {
        throw new Error('Invalid .sub file format');
      }

      const frequency = parseInt(frequencyMatch[1]);
      const raw_data = rawDataMatch[1].trim().split(/\s+/).map(Number);

      return { frequency, raw_data };
    }
  </script>
</body>
</html>
```

#### Example 3: React Component

```jsx
import React, { useState } from 'react';
import PatternDecoder from '../matcher/js/index.js';

export function SubFileDecoder() {
  const [matches, setMatches] = useState([]);
  const decoder = new PatternDecoder();

  const handleFileUpload = async (e) => {
    const file = e.target.files[0];
    const text = await file.text();
    const metadata = parseSubFile(text);
    const results = decoder.decode(metadata);
    setMatches(results);
  };

  return (
    <div>
      <input type="file" onChange={handleFileUpload} accept=".sub" />
      {matches.length > 0 && (
        <div>
          <h2>Decoded Devices ({matches.length})</h2>
          {matches.map((match, i) => (
            <div key={i} className="match-card">
              <h3>{match.name}</h3>
              <p>Manufacturer: {match.manufacturer || 'Unknown'}</p>
              <p>Category: {match.category}</p>
              <p>Confidence: {(match.confidence * 100).toFixed(1)}%</p>
              <p>Method: {match.matchMethod}</p>
            </div>
          ))}
        </div>
      )}
    </div>
  );
}
```

---

## Part 4: Implementation Roadmap

### Phase 1: Cleanup (1 day)

**Priority: Remove redundant code**

1. ✅ **Delete redundant matchers** (~2 hours)
   - Remove `strategies_orm.py`
   - Remove `simple_matcher.py`
   - Remove `rtl433_matcher.py`
   - Update imports across codebase

2. ✅ **Consolidate API** (~2 hours)
   - Rename `main_simple.py` → `app.py` (or keep as `main_simple.py`)
   - Archive old `main.py` → `main_orm_legacy.py`
   - Update `start_web.sh` to use correct file

3. ✅ **Archive old scripts** (~1 hour)
   - Create `scripts/archive/` directory
   - Move 11 one-time scripts
   - Update `scripts/README.md`

4. ✅ **Review parser usage** (~1 hour)
   - Check if `raw_parser.py`, `gps_extractor.py`, `wardriving_importer.py` are used
   - Move to `deprecated/` if unused

**Outcome:** ~1,500 lines of code removed, cleaner structure

---

### Phase 2: Fix Pattern Decoder Integration (2 hours)

**Priority: Make pattern decoder work in production**

1. ✅ **Update API to use strategies.py** (~1 hour)
   ```python
   # Replace simple_matcher with MatchEngine
   from src.matcher.engine import MatchEngine
   from src.matcher.strategies import *

   engine = MatchEngine()
   engine.add_strategy(ExactMatcher())
   engine.add_strategy(FrequencyMatcher())
   engine.add_strategy(PatternBasedStrategy())  # NEW!
   engine.add_strategy(RTL433DecoderStrategy())
   ```

2. ✅ **Test pattern decoder in API** (~1 hour)
   - Upload test .sub files
   - Verify pattern matches appear in results
   - Check confidence scores

**Outcome:** Pattern decoder fully integrated and working

---

### Phase 3: JavaScript Implementation (3-5 days)

**Priority: Port pattern decoder to JavaScript**

1. ✅ **Core implementation** (~2 days)
   - `protocol-database.js` (port 18 protocols)
   - `pulse-analyzer.js` (K-means, pulse detection)
   - `fingerprint.js` (statistics)
   - `pattern-decoder.js` (main logic)
   - `index.js` (exports)

2. ✅ **Testing** (~1 day)
   - Unit tests (Jest or Mocha)
   - Test with same .sub files as Python
   - Verify decode rate matches (44.4%)

3. ✅ **Browser integration** (~1-2 days)
   - Build .sub file parser
   - Create demo HTML page
   - Optimize for bundle size
   - Add to GigLez frontend

**Outcome:** Client-side decoding capability

---

### Phase 4: Optimization (1-2 days)

**Priority: Improve decode rate**

1. ✅ **Lower confidence thresholds** (~30 min)
2. ✅ **Add more protocols** (~2 hours)
3. ✅ **Adaptive timing tolerance** (~1 hour)
4. ✅ **Enhance pattern matching** (~2 hours)

**Target:** 60%+ decode rate (vs current 44.4%)

---

## Summary

### Code to Remove

| File | Lines | Reason |
|------|-------|--------|
| `strategies_orm.py` | 356 | Replaced by strategies.py |
| `simple_matcher.py` | 351 | Replaced by strategies.py |
| `rtl433_matcher.py` | 352 | Replaced by strategies.py |
| 11 archived scripts | ~800 | One-time use, move to archive/ |
| **Total** | **~1,859 lines** | **19% reduction** |

### Code to Add

| Component | Lines (estimate) | Purpose |
|-----------|------------------|---------|
| JavaScript protocol DB | ~300 | Client-side decoding |
| JavaScript decoder | ~400 | Core logic |
| JavaScript utilities | ~200 | K-means, fingerprinting |
| **Total** | **~900 lines** | **New capability** |

### Net Result

- **Remove:** 1,859 lines of redundant Python
- **Add:** 900 lines of new JavaScript
- **Net reduction:** 959 lines (-10%)
- **New capability:** Client-side decoding
- **Improved maintainability:** Single matcher system

---

## Next Steps

**Immediate (Today):**
1. Review this plan
2. Get approval for deletions
3. Start Phase 1: Cleanup

**Short-term (This Week):**
1. Complete Phase 1 & 2
2. Start Phase 3: JavaScript implementation

**Medium-term (Next Week):**
1. Complete JavaScript implementation
2. Test in browser
3. Optimize decoder performance

---

**Status:** Plan Complete - Ready for Implementation
**Approval Needed:** Yes (for deletions)
**Risk Level:** Low (all changes are backwards-compatible or cleanup)