From 0a40103fd5c72411047ae0f7736186c24f1715d1 Mon Sep 17 00:00:00 2001 From: n0mad1k Date: Tue, 17 Mar 2026 12:09:22 -0400 Subject: [PATCH] =?UTF-8?q?Full=20PCAP=20on=20all=20tiers=20=E2=80=94=20no?= =?UTF-8?q?=20headers-only,=20no=20tiered=20retention.=20128GB+=20with=20z?= =?UTF-8?q?std-19=20handles=20it.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- BIGBROTHER_DESIGN.md | 8 ++++---- OPERATIONAL_WORKFLOW.md | 14 ++++++-------- 2 files changed, 10 insertions(+), 12 deletions(-) diff --git a/BIGBROTHER_DESIGN.md b/BIGBROTHER_DESIGN.md index 3c6c939..fec2a43 100644 --- a/BIGBROTHER_DESIGN.md +++ b/BIGBROTHER_DESIGN.md @@ -263,8 +263,8 @@ Scope is the operator's responsibility. An optional `scope.yaml` can be loaded t - Decompression on operator's workstation: `zstd -d` then open in Wireshark - CPU cost: zstd -19 on Pi 4 compresses at ~5-10MB/s (runs as background task after rotation, not inline) - Pi Zero: use zstd -3 (fast mode, still 5-8x compression, minimal CPU) -- Snap length configurable: 96 bytes (headers only) for Pi Zero, 65535 (full) for Pi 4+ -- **Tiered retention**: Headers-only (snap=96) as default on all tiers for bulk traffic. Separate credential-protocol full-capture ring buffer (NTLM, Kerberos, HTTP auth ports only). Full PCAP is operator-opt-in for targeted sessions. A 100Mbps link generates ~45GB/hour at full snap — without tiering, 32GB fills in hours and auto-purge destroys historical data. +- **Full packet capture (snap_length=65535) on all tiers.** With 128GB+ SD and zstd -19 compression (10-15x), effective storage is ~1.2TB. Full payloads are required for file carving, email reconstruction, print job extraction, database query capture, and credential extraction from protocol payloads. Headers-only would blind most surveillance modules. +- Pi Zero 2W: full capture still applies but with fewer concurrent modules to stay within memory budget. Compression downgraded to zstd -3 to reduce CPU load. - Disk usage monitoring — auto-purge oldest PCAPs when threshold hit (bulk first, credential ring preserved) - PcapNG format support for metadata embedding (interface info, comments, timestamps) @@ -1523,7 +1523,7 @@ capture: pcap_compression: "zstd" # zstd max compression on rotation pcap_compression_level: 19 # 1-22, default 19 (max practical). Pi Zero: auto-downgrade to 3 pcap_format: "pcapng" # pcapng (preferred) or pcap - snap_length: "auto" # auto (by hardware tier) | 96 | 65535 + snap_length: 65535 # Always full capture — 128GB+ SD with zstd compression handles it bpf_filter: "full_capture" # full_capture | credentials | dns | custom custom_bpf: "" disk_max_pct: 85 # Purge oldest PCAPs above this (128GB+ gives headroom) @@ -1650,7 +1650,7 @@ When enabled: ARP spoof validates each target IP, DHCP spoof only responds to in |---|:---:|:---:|:---:| | **Max passive modules** | 4 | All 22 | All 22 | | **Max active modules** | 1 | 6 | Unlimited | -| **Snap length** | 96 bytes | 65535 | 65535 | +| **Snap length** | 65535 (full) | 65535 (full) | 65535 (full) | | **Flow table max** | 5,000 | 50,000 | 100,000 | | **PCAP compression** | zstd -3 (fast) | zstd -19 (max) | zstd -19 (max) | | **Key derivation** | pbkdf2 | argon2id | argon2id | diff --git a/OPERATIONAL_WORKFLOW.md b/OPERATIONAL_WORKFLOW.md index 1441288..256477e 100644 --- a/OPERATIONAL_WORKFLOW.md +++ b/OPERATIONAL_WORKFLOW.md @@ -23,15 +23,13 @@ After 72 hours of passive collection on this network, here is exactly what each ``` storage/pcaps/ -├── headers_2026-03-18_00-00.pcap.zst.enc (4.1MB — snap=96 headers-only) -├── headers_2026-03-18_01-00.pcap.zst.enc (3.8MB) -├── headers_2026-03-18_02-00.pcap.zst.enc (0.9MB — overnight, minimal traffic) -├── ... (24 hourly files) -├── cred_ring_2026-03-18.pcap.zst.enc (18MB — full payload, auth ports only) -└── full_capture_operator_session_1.pcap.zst.enc (operator-triggered) +├── full_2026-03-18_00-00.pcap.zst.enc (47MB — full payload, zstd -19 compressed) +├── full_2026-03-18_01-00.pcap.zst.enc (52MB) +├── full_2026-03-18_02-00.pcap.zst.enc (8MB — overnight, minimal traffic) +├── ... (24 hourly files) ``` -**Size**: ~80MB/day compressed for headers-only (from ~2.1GB raw). Credential ring (full-payload capture on ports 21, 25, 80, 88, 110, 143, 389, 445, 1433, 3306, 3389, 5432, 9100) adds ~18MB/day. At this rate, 128GB SD holds 3+ months of headers and credential traffic. +**Size**: This dental office generates ~15-25Mbps average traffic during business hours, near-zero overnight. That's roughly 8-12GB raw per day. With zstd -19 compression (10-15x on network PCAPs): **~600MB-1.2GB/day compressed**. On a 128GB SD card, that's **100-200 days** of full capture. On a 256GB card, over a year. Full payloads mean every print job, every SQL query, every file transfer, every cleartext credential is in the PCAP for offline analysis. **Protocol breakdown for this network (24h, by packet count)**: @@ -49,7 +47,7 @@ ARP 2% Normal L2 resolution Other 2% NTP, ICMP, LLMNR, NetBIOS ``` -**What you can do with it**: Open the credential ring PCAPs on your workstation in Wireshark to manually inspect any authentication you want to re-examine. The headers-only PCAPs give you flow-level forensics (who connected to whom, when, how much data) without the storage cost of full payload. +**What you can do with it**: Pull the compressed PCAPs to your workstation (`zstd -d` to decompress), open in Wireshark for full protocol analysis. Every packet payload is there — you can reconstruct files, replay sessions, extract credentials you missed, carve attachments. Full PCAP is the ground truth that backs up every other module's findings. ---