minor updates

This commit is contained in:
KaraZajac
2026-02-17 22:37:39 -05:00
parent 4bc7bd7beb
commit 77547bb437
16 changed files with 222 additions and 6 deletions
+104
View File
@@ -0,0 +1,104 @@
# Sub Decode: ProtoPirate vs KAT
Comparison of how **ProtoPirate** (REFERENCES/protopirate) and **KAT** decode Flipper `.sub` (RAW) files.
## File format
Both use the same Flipper SubGhz RAW format:
- **Filetype:** `Flipper SubGhz RAW File`
- **Protocol:** `RAW`
- **Frequency:** one value (Hz)
- **RAW_Data:** space-separated **int32** values:
- **Positive** = HIGH level
- **Negative** = LOW level
- **Magnitude** = duration in **microseconds**
So parsing and (level, duration) stream content are the same.
---
## ProtoPirate sub decode (REFERENCES/protopirate)
### Where it lives
- **Scene:** `scenes/protopirate_scene_sub_decode.c`
- **Raw reader:** `helpers/raw_file_reader.c` / `raw_file_reader.h`
### Flow
1. **Open file**
`raw_file_reader_open()` uses FlipperFormat to open the file, checks header "Flipper SubGhz RAW File" and Protocol "RAW". Does **not** read Frequency in the reader; the scene reads that separately.
2. **Read metadata (scene)**
In `DecodeStateReadHeader` / `DecodeStateStartingWorker` the scene opens the file again with FlipperFormat and reads:
- **Frequency** (default 433920000 if missing)
- **Preset** (e.g. AM650, FM238) and maps it to the SubGhz preset used for the receiver.
3. **Feed stream**
In `DecodeStateDecodingRaw`:
- Loop: `raw_file_reader_get_next(ctx->raw_reader, &level, &duration)` to get the next (level, duration).
- For each pair: **`subghz_receiver_decode(app->txrx->receiver, level, duration)`**.
- Reads in chunks of **128** samples per tick (`SAMPLES_TO_READ_PER_TICK`) for UI responsiveness.
4. **On decode**
When the Flipper receiver reports a decode (`protopirate_sub_decode_receiver_callback`):
- Add the decode to history.
- **`subghz_receiver_reset(receiver)`** so all decoders are reset.
- Continue feeding the **same** file from the next sample.
So: one continuous stream from the file, (level, duration) in order; on each decode → record → reset receiver → keep going. **No** polarity inversion in this path. **No** sliding window or multiple start positions.
### Raw file reader details
- **`raw_file_reader_get_next()`** (raw_file_reader.c):
Reads next int32 from buffer; if buffer is empty, loads next chunk via `flipper_format_read_int32(..., "RAW_Data", ...)`.
`level = (value >= 0)`, `duration = abs(value)`. Same convention as KAT.
---
## KAT sub import (src/app.rs + protocols/mod.rs + export/flipper.rs)
### Where it lives
- **Import:** `src/export/flipper.rs``import_sub_raw(path)` → returns `(frequency, Vec<StoredLevelDuration>)`.
- **Decode:** `src/protocols/mod.rs``process_signal_stream()` / `process_signal_stream_inner()`.
- **Use:** `src/app.rs` → when loading a .sub file, calls `import_sub_raw` then `protocols.process_signal_stream(&pairs, frequency)`.
### Flow
1. **Parse file**
`import_sub_raw()`:
- Reads whole file as text.
- Parses **Frequency** (default 433_920_000 if missing).
- Parses all **RAW_Data** lines into one list of (level, duration) with the same rule: positive ⇒ HIGH, negative ⇒ LOW, duration = abs(value) µs.
2. **Decode stream**
`process_signal_stream(pairs, frequency)`:
- Tries **normal polarity** first: `process_signal_stream_inner(pairs, frequency, false)`.
- If that returns **no** decodes, tries **inverted polarity**: `process_signal_stream_inner(pairs, frequency, true)` (flip level for every pair).
- Inner loop: for each (level, duration) in order, for each decoder that supports the file frequency, call **`decoder.feed(level, duration_us)`**. If any decoder returns a decode:
- Record (protocol name, decoded signal, segment of pairs).
- **Reset all decoders** and set `segment_start` to next index.
- Continue from the next pair.
So: one pass over the in-memory stream; on each decode → record → reset all decoders → continue. **Difference:** KAT also runs a **second** pass with **inverted polarity** if the first pass finds nothing.
---
## Differences summary
| Aspect | ProtoPirate | KAT |
|--------|-------------|-----|
| **File format** | Same (Flipper RAW, positive=HIGH, negative=LOW, µs) | Same |
| **Stream order** | Same (sequential level/duration from file) | Same |
| **Frequency** | Read from file, used for preset/receiver | Read from file, used for decoder filter (2% tolerance) |
| **Decode loop** | One (level, duration) at a time → `subghz_receiver_decode()` | One (level, duration) at a time → `decoder.feed()` for each decoder |
| **On decode** | Add to history, **subghz_receiver_reset()**, continue same file | Append to list, **reset all decoders**, continue same stream |
| **Polarity** | Single polarity (as in file) | Tries **normal**, then **inverted** if no decodes |
| **Sliding window / multiple starts** | No | No |
| **“No protocol” result** | Shows “No ProtoPirate protocol detected” (no “Unknown” capture) | No capture if no decoder ever returns (same idea) |
So the **sub decode strategy is the same**: single stream, reset-after-each-decode, no sliding window. The only functional difference is **KATs extra inverted-polarity pass** when the normal pass finds no decodes.
For a file like the VAG Suran .sub: the first HIGH pulse is 133 µs, so VAGs Reset condition (300±79 or 500±79 µs) is never met. In both codebases the decoder would stay in Reset and never emit a decode; neither would create a capture from that stream. ProtoPirate would show “No ProtoPirate protocol detected”; KAT would add no capture. So for that case the behavior is aligned; fixing it would require something like trying decode from multiple start indices (sliding window) or trimming to the first VAG-like preamble, in either codebase.
+70 -6
View File
@@ -108,21 +108,66 @@ def analyze_reset_and_first_steps(pairs: list[tuple[bool, int]], invert: bool) -
return lines return lines
def find_first_vag_like_pulse(pairs: list[tuple[bool, int]], invert: bool) -> list[str]: def find_first_vag_like_pulse(
"""Find first HIGH pulse that looks like VAG preamble (300±79 or 500±79).""" pairs: list[tuple[bool, int]], invert: bool, tolerance_300: int = 79, tolerance_500: int = 79
) -> list[str]:
"""Find first HIGH pulse that looks like VAG preamble (300±tolerance or 500±tolerance)."""
lines: list[str] = [] lines: list[str] = []
for i, (level, duration) in enumerate(pairs): for i, (level, duration) in enumerate(pairs):
if invert: if invert:
level = not level level = not level
if not level: if not level:
continue continue
ok_300 = (TE_SHORT_12 - REF_RESET_DELTA <= duration <= TE_SHORT_12 + REF_RESET_DELTA) ok_300 = (TE_SHORT_12 - tolerance_300 <= duration <= TE_SHORT_12 + tolerance_300)
ok_500 = (TE_SHORT - REF_RESET_DELTA <= duration <= TE_SHORT + REF_RESET_DELTA) ok_500 = (TE_SHORT - tolerance_500 <= duration <= TE_SHORT + tolerance_500)
if ok_300 or ok_500: if ok_300 or ok_500:
kind = "300±79 (Type1/2)" if ok_300 else "500±79 (Type3/4)" kind = f"300±{tolerance_300} (Type1/2)" if ok_300 else f"500±{tolerance_500} (Type3/4)"
lines.append(f"First VAG-like HIGH pulse at index {i}: {duration} µs ({kind})") lines.append(f"First VAG-like HIGH pulse at index {i}: {duration} µs ({kind})")
return lines return lines
lines.append("No HIGH pulse in the entire file matches 300±79 or 500±79 µs.") lines.append(f"No HIGH pulse matches 300±{tolerance_300} or 500±{tolerance_500} µs.")
return lines
def scan_vag_preamble_with_tolerances(pairs: list[tuple[bool, int]], invert: bool) -> list[str]:
"""
Scan entire file for HIGH pulses that could be VAG preamble (300 or 500 µs)
with multiple tolerances. Reports counts and first few indices for each.
"""
lines: list[str] = []
# Tolerances to try: current (79), then relaxed
tolerances = [79, 100, 120, 150, 200]
for tol in tolerances:
indices_300: list[int] = []
indices_500: list[int] = []
for i, (level, duration) in enumerate(pairs):
if invert:
level = not level
if not level:
continue
if TE_SHORT_12 - tol <= duration <= TE_SHORT_12 + tol:
indices_300.append(i)
if TE_SHORT - tol <= duration <= TE_SHORT + tol:
indices_500.append(i)
lines.append(f" Tolerance ±{tol} µs: {len(indices_300)} pulses near 300 µs, {len(indices_500)} near 500 µs")
if indices_300:
first_few = indices_300[:5]
lines.append(f" First 300±{tol} at indices: {first_few}")
if indices_500:
first_few = indices_500[:5]
lines.append(f" First 500±{tol} at indices: {first_few}")
return lines
def histogram_high_pulses(pairs: list[tuple[bool, int]], invert: bool, buckets: list[tuple[int, int]]) -> list[str]:
"""Count HIGH pulse durations in buckets (center, half_width) -> (min, max) µs."""
lines: list[str] = []
counts: list[tuple[str, int]] = []
for center, half in buckets:
lo, hi = center - half, center + half
n = sum(1 for level, d in pairs if (level if not invert else not level) and lo <= d <= hi)
counts.append((f"{center}±{half}", n))
for label, n in counts:
lines.append(f" {label} µs: {n} HIGH pulses")
return lines return lines
@@ -166,6 +211,25 @@ def main() -> None:
for line in find_first_vag_like_pulse(pairs, invert=True): for line in find_first_vag_like_pulse(pairs, invert=True):
print(line) print(line)
# Scan entire file with multiple tolerances (is there any VAG preamble at all?)
print("\n--- VAG preamble scan: counts at different tolerances ---")
print("Normal polarity:")
for line in scan_vag_preamble_with_tolerances(pairs, invert=False):
print(line)
print("Inverted polarity:")
for line in scan_vag_preamble_with_tolerances(pairs, invert=True):
print(line)
# Histogram of HIGH pulse durations (VAG uses 300, 500, 600, 1000 µs)
print("\n--- Histogram of HIGH pulse durations (µs) ---")
buckets = [(300, 79), (500, 79), (600, 79), (1000, 79), (300, 150), (500, 150)]
print("Normal polarity:")
for line in histogram_high_pulses(pairs, invert=False, buckets=buckets):
print(line)
print("Inverted polarity:")
for line in histogram_high_pulses(pairs, invert=True, buckets=buckets):
print(line)
# Summary # Summary
print("\n--- Summary (why no decode / no Unknown) ---") print("\n--- Summary (why no decode / no Unknown) ---")
if not ok: if not ok:
+25
View File
@@ -1119,6 +1119,13 @@ impl App {
} }
/// Generate a default export filename (without extension) for a capture /// Generate a default export filename (without extension) for a capture
/// Path relative to the import directory for display; falls back to full path if not under import_dir.
fn path_relative_to_import(path: &std::path::Path, import_dir: &std::path::Path) -> String {
path.strip_prefix(import_dir)
.map(|p| p.to_string_lossy().to_string())
.unwrap_or_else(|_| path.to_string_lossy().to_string())
}
fn default_export_filename(capture: &Capture) -> String { fn default_export_filename(capture: &Capture) -> String {
format!( format!(
"{}_{}", "{}_{}",
@@ -1387,6 +1394,7 @@ impl App {
// Deduplicate: same signal can decode at multiple stream positions (e.g. Ford V0 across bursts) // Deduplicate: same signal can decode at multiple stream positions (e.g. Ford V0 across bursts)
let mut seen: std::collections::HashSet<(String, u64, Option<u32>, Option<u8>)> = let mut seen: std::collections::HashSet<(String, u64, Option<u32>, Option<u8>)> =
std::collections::HashSet::new(); std::collections::HashSet::new();
let mut any_decoded_added = false;
for (protocol_name, decoded, segment_pairs) in decoded_list { for (protocol_name, decoded, segment_pairs) in decoded_list {
let key = ( let key = (
protocol_name.clone(), protocol_name.clone(),
@@ -1427,11 +1435,26 @@ impl App {
if research_mode || capture.protocol.is_some() { if research_mode || capture.protocol.is_some() {
if !self.capture_duplicate_of_existing(&capture) { if !self.capture_duplicate_of_existing(&capture) {
self.next_capture_id += 1; self.next_capture_id += 1;
capture.source_file = Some(Self::path_relative_to_import(path, self.storage.import_dir()));
self.captures.push(capture); self.captures.push(capture);
imported += 1; imported += 1;
any_decoded_added = true;
} }
} }
} }
// When no protocol decoded the stream, add a single Unknown capture if research_mode (same as live capture).
if !any_decoded_added && research_mode && !raw_pairs.is_empty() {
let mut capture = crate::capture::Capture::from_pairs_with_rf(
self.next_capture_id,
frequency,
raw_pairs.clone(),
None,
);
self.next_capture_id += 1;
capture.source_file = Some(Self::path_relative_to_import(path, self.storage.import_dir()));
self.captures.push(capture);
imported += 1;
}
} }
Err(e) => tracing::warn!("Failed to import {:?}: {}", path, e), Err(e) => tracing::warn!("Failed to import {:?}: {}", path, e),
} }
@@ -1439,6 +1462,7 @@ impl App {
match crate::export::fob::import_fob(path, self.next_capture_id) { match crate::export::fob::import_fob(path, self.next_capture_id) {
Ok(mut capture) => { Ok(mut capture) => {
self.next_capture_id += 1; self.next_capture_id += 1;
capture.source_file = Some(Self::path_relative_to_import(path, self.storage.import_dir()));
// Re-run decoder when Unknown and raw_pairs present (same as .sub) // Re-run decoder when Unknown and raw_pairs present (same as .sub)
if capture.status == crate::capture::CaptureStatus::Unknown if capture.status == crate::capture::CaptureStatus::Unknown
&& !capture.raw_pairs.is_empty() && !capture.raw_pairs.is_empty()
@@ -1638,6 +1662,7 @@ impl App {
make: None, make: None,
model: None, model: None,
region: None, region: None,
source_file: None,
}; };
self.next_capture_id += 1; self.next_capture_id += 1;
self.captures.push(capture); self.captures.push(capture);
+4
View File
@@ -76,6 +76,9 @@ pub struct Capture {
/// Region (e.g. NA, EU) for vulnerability lookup and .fob export. /// Region (e.g. NA, EU) for vulnerability lookup and .fob export.
#[serde(default)] #[serde(default)]
pub region: Option<String>, pub region: Option<String>,
/// Source file path when imported from .sub or .fob; None for live captures.
#[serde(default)]
pub source_file: Option<String>,
} }
/// Modulation type used by protocol (encoding: PWM vs Manchester) /// Modulation type used by protocol (encoding: PWM vs Manchester)
@@ -154,6 +157,7 @@ impl Capture {
make: None, make: None,
model: None, model: None,
region: None, region: None,
source_file: None,
} }
} }
+2
View File
@@ -306,6 +306,7 @@ fn import_fob_v2(fob: &FobFile, next_id: u32) -> Result<Capture> {
make: None, make: None,
model: None, model: None,
region: None, region: None,
source_file: None,
}) })
} }
@@ -376,6 +377,7 @@ fn import_fob_v1(fob: &FobFileV1, next_id: u32) -> Result<Capture> {
make: None, make: None,
model: None, model: None,
region: None, region: None,
source_file: None,
}) })
} }
+17
View File
@@ -271,6 +271,23 @@ fn render_signal_detail(frame: &mut Frame, area: Rect, capture: &crate::capture:
Span::styled(raw_info, raw_style), Span::styled(raw_info, raw_style),
])); ]));
// Row 8: File path (imported .sub/.fob only; blank for live captures)
let file_display = capture
.source_file
.as_deref()
.unwrap_or("");
left_lines.push(Line::from(vec![
Span::styled(" File: ", label_style),
Span::styled(
file_display,
if file_display.is_empty() {
Style::default().fg(Color::DarkGray)
} else {
value_style
},
),
]));
// Build the title // Build the title
let title = format!( let title = format!(
" Signal #{:02}{} ", " Signal #{:02}{} ",