Initial public release
Full BigBrother network implant - passive SOC + active exploitation. Personal identifiers removed; all capabilities intact. See README.md for setup and docs/deployment.md for detailed deployment.
This commit is contained in:
@@ -0,0 +1,277 @@
|
||||
---
|
||||
agent: fix-planner
|
||||
status: COMPLETE
|
||||
timestamp: 2026-04-10T13:45:00Z
|
||||
total_findings_raw: 32
|
||||
total_findings_deduped: 29
|
||||
p1_count: 4
|
||||
p2_count: 6
|
||||
p3_count: 12
|
||||
p4_count: 7
|
||||
devtrack_items_created: [551, 552, 553, 554, 555, 556, 557, 558, 559, 560]
|
||||
errors: []
|
||||
---
|
||||
|
||||
# Presence Daemon Fix Plan
|
||||
|
||||
## Deduplication Notes
|
||||
- **Raw findings:** 32 across 6 audit files (code-auditor, security-auditor, env-validator, db-auditor, test-runner, impl notes)
|
||||
- **Deduped:** 29 unique issues (merged findings with same file + issue type; highest severity retained; sources cited)
|
||||
- **Sources combined:** DB connection leak (2 sources), struct unpacking edge cases (2 sources), BLE error recovery (2 sources)
|
||||
|
||||
---
|
||||
|
||||
## P1 — Block Deploy (CRITICAL/HIGH Security or Correctness)
|
||||
|
||||
- [x] **[CRITICAL]** `presence_daemon.py:648-656` HTTP status server port hardcoded (9191) — acts as fingerprint | Sources: security-auditor | Effort: XS | DevTrack: #551
|
||||
- **Context:** Port discoverable via `ss -tlnp`, identifies tool via process enumeration
|
||||
- **Fix:** Make configurable via `PRESENCE_STATUS_PORT` env var; document per-deployment variation
|
||||
- **Acceptance:** Port reads from env var; default fallback to 9191 if unset
|
||||
- **Status:** FIXED — STATUS_PORT = int(os.getenv("PRESENCE_STATUS_PORT", "9191")) added at L31; status_server() updated to use STATUS_PORT; install.sh comments updated
|
||||
|
||||
- [x] **[CRITICAL]** `presence_daemon.py:156, 172, 193, 208` SQLite connection leak in exception handlers | Sources: db-auditor | Effort: S | DevTrack: #552
|
||||
- **Context:** Functions `lookup_person()`, `seed_persons_from_db()`, `log_signal()`, `log_presence_change()` don't close connections on exceptions; resource exhaustion under high-frequency signal logging
|
||||
- **Fix:** Wrap all `sqlite3.connect()` in try/finally or use context manager (`with` statement)
|
||||
- **Acceptance:** All four functions use context manager; no unclosed connection paths
|
||||
- **Status:** FIXED — All four functions wrapped with try/finally and conn.close() in finally block; timeout increased to 10s
|
||||
|
||||
- [x] **[HIGH]** `presence_daemon.py:272-323` Race condition in `update_person_state()` — two-lock acquisition | Sources: code-auditor | Effort: S | DevTrack: #553
|
||||
- **Context:** Lock released between `devices_lock` and `persons_lock`; `max_last_seen` snapshot becomes stale. Concurrent signal could change person state after certainty computed but before lock acquired, causing duplicate alerts or missed transitions.
|
||||
- **Fix:** Acquire both locks atomically or compute entire state transition inside single critical section; snapshot under lock before computing state machine
|
||||
- **Acceptance:** State machine transitions (UNKNOWN→PRESENT, PRESENT→ABSENT) protected by single lock; no log entries showing duplicate/missed transitions
|
||||
- **Status:** FIXED — Certainty computed under devices_lock and snapshot held; state machine logic moved inside persons_lock; alert sending moved outside locks to minimize critical section
|
||||
|
||||
- [ ] **[HIGH]** `presence_daemon.py:384-404` Raw socket struct unpacking lacks full validation | Sources: security-auditor | Effort: S | DevTrack: #554
|
||||
- **Context:** Netlink parsing checks `nlmsg_len < 16` but not actual buffer length; DHCP MAC extraction at `[28:34]` not validated. Crafted packets could cause silent truncation, incorrect MAC parsing, or exception flood.
|
||||
- **Fix:** Add explicit length checks before struct.unpack_from() and MAC extraction:
|
||||
```python
|
||||
# Netlink: before struct.unpack_from('=IHHII', data, offset)
|
||||
if len(data) < offset + 16:
|
||||
logger.warning("Truncated netlink message")
|
||||
return
|
||||
|
||||
# DHCP: before mac_bytes = dhcp[28:34]
|
||||
if len(dhcp) < 34:
|
||||
return
|
||||
```
|
||||
- **Acceptance:** All edge cases validated; unit tests (if added) should cover truncated packets
|
||||
- **Status:** NOT FIXED (out of scope for this bugfixer task — only 4 P1 items assigned)
|
||||
|
||||
---
|
||||
|
||||
## P2 — Fix This Week (HIGH Code Quality or Significant Risk)
|
||||
|
||||
- [ ] **[HIGH]** `presence_daemon.py:289` Magic number in aggregate certainty calculation | Sources: code-auditor | Effort: XS | DevTrack: #555
|
||||
- **Context:** Hardcoded `0.08` coefficient for secondary device weighting (line 289: `certainties[0] + 0.08 * certainties[1]`) should be named constant for clarity and maintainability
|
||||
- **Fix:** Extract to module-level constant:
|
||||
```python
|
||||
SECONDARY_DEVICE_WEIGHT = 0.08
|
||||
# Then in update_person_state():
|
||||
aggregate_certainty = certainties[0] + SECONDARY_DEVICE_WEIGHT * certainties[1]
|
||||
```
|
||||
- **Acceptance:** Constant named, value unchanged, formula still produces same results
|
||||
|
||||
- [ ] **[HIGH]** `presence_daemon.py:20` Unused import urllib.parse | Sources: code-auditor | Effort: XS | DevTrack: #556
|
||||
- **Context:** Imported but never used; only `urllib.request.Request()` and `urllib.request.urlopen()` are called
|
||||
- **Fix:** Remove line 20: `import urllib.parse`
|
||||
- **Acceptance:** No import errors; linter reports no unused imports
|
||||
|
||||
- [ ] **[HIGH]** `install.sh:31` Database path mismatch (install.sh vs daemon defaults) | Sources: code-auditor | Effort: XS | DevTrack: #557
|
||||
- **Context:** install.sh forces `/opt/sensor/sensor.db` (line 31) but daemon defaults to `/opt/presence/presence.db` (presence_daemon.py:29). Will create two separate databases.
|
||||
- **Fix:** Align paths. Recommend standardizing on `/opt/presence/presence.db` (more descriptive name):
|
||||
- Update install.sh line 31: `PRESENCE_DB_PATH=/opt/presence/presence.db`
|
||||
- Verify daemon defaults match or accept via env var override
|
||||
- **Acceptance:** Single database file created; systemd EnvironmentFile sets consistent path; no duplicate databases on disk
|
||||
|
||||
- [ ] **[HIGH]** `presence_daemon.py:541-573` BLE scanner missing error recovery — event loop can die silently | Sources: security-auditor, db-auditor | Effort: M | DevTrack: #558
|
||||
- **Context:** No retry loop or watchdog; malformed BLE packets could crash event loop, silently disabling BLE tracking. Also overlaps with seed_persons_from_db() error handling (db-auditor: missing recovery on empty persons table).
|
||||
- **Fix:**
|
||||
1. Add retry loop with exponential backoff to `ble_scanner()` thread:
|
||||
```python
|
||||
retries = 0
|
||||
while retries < 5:
|
||||
try:
|
||||
loop.run_until_complete(run_ble_scan())
|
||||
retries = 0 # Reset on success
|
||||
except Exception as e:
|
||||
logger.error(f"BLE scan iteration failed: {e}")
|
||||
retries += 1
|
||||
time.sleep(2 ** retries) # Exponential backoff: 2, 4, 8, 16, 32 sec
|
||||
logger.critical("BLE scanner exhausted retries; disabling")
|
||||
```
|
||||
2. Implement watchdog thread to detect BLE thread death and restart
|
||||
3. Validate persons table on startup in `seed_persons_from_db()`: fail loud if empty and no override
|
||||
- **Acceptance:** BLE scanner recovers from transient errors; critical log on exhaustion; daemon validates person list at startup
|
||||
|
||||
- [x] **[HIGH]** `presence_daemon.py:142-147` init_db() missing WAL mode and pragmas | Sources: db-auditor | Effort: XS | DevTrack: #559
|
||||
- **Context:** No PRAGMA journal_mode=WAL; synchronous journaling under 6-thread concurrent write load will serialize and timeout (SQLITE_BUSY errors)
|
||||
- **Fix:** Add to `init_db()` or presence_schema.sql:
|
||||
```python
|
||||
# In init_db(), after conn.execute(schema):
|
||||
conn.execute('PRAGMA journal_mode=WAL')
|
||||
conn.execute('PRAGMA synchronous=NORMAL')
|
||||
conn.execute('PRAGMA foreign_keys=ON')
|
||||
conn.commit()
|
||||
```
|
||||
- **Acceptance:** `PRAGMA journal_mode` reports WAL; no SQLITE_BUSY timeouts under sustained signal load; schema integrity enforced
|
||||
- **Status:** FIXED — WAL pragmas added to init_db() at L148-150, committed at L151
|
||||
|
||||
- [ ] **[HIGH]** `presence_daemon.py:605` N+1 pattern in decay_loop — inefficient person state recomputation | Sources: db-auditor | Effort: M | DevTrack: #560
|
||||
- **Context:** Every person state update re-computes aggregate certainty from full devices dict; should batch-compute all person states once per decay interval
|
||||
- **Fix:** Refactor decay_loop to compute all person states in one pass:
|
||||
```python
|
||||
def decay_loop():
|
||||
while True:
|
||||
time.sleep(60)
|
||||
try:
|
||||
with devices_lock:
|
||||
# Collect all person_ids
|
||||
person_ids = set(d.person_id for d in devices.values() if d.person_id)
|
||||
# Compute all person states in one critical section
|
||||
decayed_devices = {mac: decay_device(dev) for mac, dev in devices.items()}
|
||||
|
||||
# State transitions outside lock
|
||||
for person_id in person_ids:
|
||||
update_person_state(person_id)
|
||||
```
|
||||
- **Acceptance:** Decay loop computes person states once per 60-second interval; no duplicate iterations per person
|
||||
|
||||
---
|
||||
|
||||
## P3 — Fix This Month (MEDIUM Code Quality or Noticeable Risk)
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:468-538` High cyclomatic complexity in parse_dhcp() | Sources: code-auditor | Effort: S
|
||||
- Description: 71 lines, 20 branches, 5 nesting levels; mixes header parsing + DHCP option parsing
|
||||
- Fix: Extract packet validation to separate function; move DHCP option parsing to dedicated helper
|
||||
- Remediation: `parse_dhcp_options(dhcp_payload) -> dict` returning option_type → value mapping
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:326-361` Deep nesting in probe_sniffer() — 6 levels | Sources: code-auditor | Effort: S
|
||||
- Description: Multiple if statements nested 6 levels; hard to follow control flow
|
||||
- Fix: Extract frame validation logic into `validate_probe_request(data) -> Optional[str]` returning MAC or None
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:576-609` Inconsistent error handling in decay_loop() | Sources: code-auditor | Effort: S
|
||||
- Description: Catches all exceptions but silently continues; no distinction between transient and fatal errors
|
||||
- Fix: Separate transient network errors (debug level) from state machine errors (error/warning level)
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:282-284` Early return in update_person_state() leaves persons_lock unprotected | Sources: code-auditor | Effort: XS
|
||||
- Description: If no devices found, returns inside devices_lock but person state not re-checked
|
||||
- Fix: Re-check person state after lock release or defer return until both locks acquired
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:437-439` Weak infrastructure IP filtering — empty set never populated | Sources: code-auditor | Effort: S
|
||||
- Description: infrastructure_ips checked but never initialized; filter never applies
|
||||
- Fix: Either populate from config at startup or remove dead code
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:661-686` Daemon thread join() semantics — daemon=True threads not blocking | Sources: code-auditor | Effort: S
|
||||
- Description: All threads created with daemon=True then joined; join() is vestigial
|
||||
- Fix: Remove daemon=True flag (prefer explicit cleanup) or remove join() calls
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:43-50` Logging fingerprints — tool identity in log messages | Sources: security-auditor | Effort: S
|
||||
- Description: Messages like "Sensor daemon started", "Probe sniffer", "DHCP sniffer" identify daemon as surveillance tool
|
||||
- Fix: Use generic messages ("Network listener probe active", "Listening for ARP events"); move identification to comments only
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:296-323` HTTP webhook timeout under lock — blocks person state queries | Sources: security-auditor | Effort: S
|
||||
- Description: persons_lock held during send_alert(); 5-second HTTP timeout blocks all state reads
|
||||
- Fix: Snapshot person data under lock, release before send_alert():
|
||||
```python
|
||||
with persons_lock:
|
||||
person = persons.get(person_id)
|
||||
if person is None:
|
||||
return
|
||||
old_status = person.status
|
||||
# ... state machine ...
|
||||
person.status = new_status
|
||||
|
||||
# Outside lock
|
||||
send_alert(person.name, "PRESENT", ...)
|
||||
```
|
||||
|
||||
- [ ] **[MEDIUM]** `install.sh:20-35` Install script missing Matrix webhook configuration | Sources: env-validator | Effort: S
|
||||
- Description: Systemd unit does not include PRESENCE_MATRIX_WEBHOOK; users must manually add
|
||||
- Fix: Add Infisical integration; create start.sh wrapper that fetches webhook from vault at boot
|
||||
- Remediation: `eval $(creds env bigbrother)` pattern in start.sh
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_schema.sql:22` Missing index on occupancy_log(ts) | Sources: db-auditor | Effort: XS
|
||||
- Description: High-frequency writes without ts index; time-range queries will full-table scan
|
||||
- Fix: Add `CREATE INDEX idx_occupancy_ts ON occupancy_log(ts);`
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py:156, 172, 193, 208` Connection timeout too short (1 second) | Sources: db-auditor | Effort: XS
|
||||
- Description: Under multi-threaded contention + synchronous journaling, SQLITE_BUSY errors likely
|
||||
- Fix: Increase timeout to 10-30 seconds in all sqlite3.connect(timeout=...) calls OR enable WAL + NORMAL synchronous (via P2 #559)
|
||||
|
||||
- [ ] **[MEDIUM]** `presence_daemon.py` No tests exist for presence daemon | Sources: test-runner | Effort: L
|
||||
- Description: Zero test coverage for 690-line daemon with 15+ functions
|
||||
- Note: Per task instructions, tests not created by test-runner. Flag for human review if required as P1 acceptance criterion.
|
||||
|
||||
---
|
||||
|
||||
## P4 — Backlog (LOW Code Quality or Style)
|
||||
|
||||
- [ ] **[LOW]** `presence_daemon.py:601` Inefficient device pruning during iteration | Effort: XS
|
||||
- Description: Deletes from dict during iteration (safe via deferred list but inelegant)
|
||||
- Fix: Build new dict with comprehension: `devices = {mac: dev for mac, dev in devices.items() if elapsed_minutes < 120}`
|
||||
|
||||
- [ ] **[LOW]** `presence_daemon.py:612-614` Inline handler factory pattern creates class per request | Effort: XS
|
||||
- Description: StatusHandler class defined in function, returns new class per request
|
||||
- Fix: Define StatusHandler at module level, pass to HTTPServer
|
||||
|
||||
- [ ] **[LOW]** `presence_daemon.py:241-270` Bare on_signal() calls lack source context | Effort: S
|
||||
- Description: Called from 5 sources but doesn't log caller/thread context
|
||||
- Fix: Add logger context or thread names for debugging multi-threaded flow
|
||||
|
||||
- [ ] **[LOW]** `presence_schema.sql:1-34` No compound index on frequent queries | Effort: XS
|
||||
- Description: idx_signals_mac and idx_signals_ts separate but queries filter on both
|
||||
- Fix: Add `CREATE INDEX idx_signals_mac_ts ON signals(mac, ts);`
|
||||
|
||||
- [ ] **[LOW]** `install.sh:10-11` Unnecessary chown after mkdir | Effort: XS
|
||||
- Description: mkdir with sudo, then chown may fail or be no-op
|
||||
- Fix: Run entire install non-root with sudo only for systemd registration
|
||||
|
||||
- [ ] **[LOW]** `presence_schema.sql:1-28` Schema lacks enforced foreign keys | Effort: XS
|
||||
- Description: REFERENCES present but PRAGMA foreign_keys never enabled
|
||||
- Fix: Execute `PRAGMA foreign_keys=ON;` in init_db() before schema execution
|
||||
|
||||
- [ ] **[LOW]** `presence_daemon.py:142-147` No connection validation in init_db() | Effort: S
|
||||
- Description: Never verifies schema creation succeeded or tables readable
|
||||
- Fix: Add startup check to validate tables exist and are readable
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Priority | Count | Effort | DevTrack | Status |
|
||||
|----------|-------|--------|----------|--------|
|
||||
| P1 | 4 | 3×S, 1×XS | #551-554 | BLOCKED until fixed |
|
||||
| P2 | 6 | 2×M, 3×S, 1×XS | #555-560 | BLOCKED until fixed |
|
||||
| P3 | 12 | 1×L, 8×S, 3×XS | — | Can run tests after P1+P2 |
|
||||
| P4 | 7 | 1×S, 6×XS | — | Nice to have |
|
||||
|
||||
---
|
||||
|
||||
## Effort Breakdown
|
||||
- **XS** (<30 min): 8 items (magic number, unused import, path mismatch, connection timeout, occupancy index, compound index, chown, foreign keys)
|
||||
- **S** (30 min - 2 hr): 12 items (connection leak, race condition, struct validation, error handling, logging, webhook, deep nesting, nesting, etc.)
|
||||
- **M** (2 - 8 hr): 2 items (BLE recovery, N+1 pattern, WAL mode + pragmas)
|
||||
- **L** (1 - 3 days): 1 item (test suite)
|
||||
|
||||
**Minimal path to unblock deploy (P1+P2):** ~10 hours effort across 10 critical fixes.
|
||||
|
||||
---
|
||||
|
||||
## Deployment Gate
|
||||
|
||||
**Item #530 cannot advance to testing until:**
|
||||
1. All P1 items (#551-554) fixed and verified
|
||||
2. All P2 items (#555-560) fixed and verified
|
||||
3. Code review confirms no regressions in:
|
||||
- Signal processing pipeline
|
||||
- Person state machine transitions
|
||||
- Database persistence and recovery
|
||||
- Thread safety (no new deadlocks)
|
||||
|
||||
**Testing verification checklist:**
|
||||
- [ ] No SQLITE_BUSY errors under sustained signal load
|
||||
- [ ] No connection exhaustion after 1 hour runtime
|
||||
- [ ] Person state transitions fire correctly (no duplicates, no missed alerts)
|
||||
- [ ] Status endpoint accessible on configurable port
|
||||
- [ ] Database path matches install.sh defaults or env override
|
||||
- [ ] BLE scanner recovers from transient errors
|
||||
@@ -0,0 +1,202 @@
|
||||
# Presence Daemon — Person Presence Intelligence
|
||||
|
||||
Passive multi-signal WiFi/ARP/DHCP/BLE person presence tracking daemon for BigBrother drop implant.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
cd /path/to/bigbrother/presence
|
||||
sudo bash install.sh
|
||||
```
|
||||
|
||||
This creates `/opt/sensor/sensor.py`, initializes the database, and starts the systemd service.
|
||||
|
||||
## Configuration
|
||||
|
||||
Environment variables:
|
||||
- `PRESENCE_MONITOR_IFACE` — WiFi monitor mode interface (default: wlan1)
|
||||
- `PRESENCE_DATA_IFACE` — Connected network interface for ARP/DHCP (default: wlan0)
|
||||
- `PRESENCE_DB_PATH` — SQLite database location (default: /opt/presence/presence.db)
|
||||
- `PRESENCE_MATRIX_WEBHOOK` — Matrix webhook URL for alerts (default: empty, no alerts)
|
||||
|
||||
Example systemd override:
|
||||
```bash
|
||||
sudo systemctl edit sensor
|
||||
# Add to [Service] section:
|
||||
# Environment=PRESENCE_MONITOR_IFACE=wlan2
|
||||
# Environment=PRESENCE_MATRIX_WEBHOOK=https://matrix.example.com/hook/...
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Sensor Threads (4 concurrent)
|
||||
|
||||
1. **Probe Sniffer** — WiFi probe request capture on monitor interface
|
||||
- Passive 802.11 frame parsing (type 0, subtype 4)
|
||||
- Extracts source MAC → +0.60 certainty bump
|
||||
- Runs on PRESENCE_MONITOR_IFACE
|
||||
|
||||
2. **ARP Listener** — netlink RTM_NEWNEIGH/RTM_DELNEIGH events
|
||||
- Kernel neighbor discovery events
|
||||
- Filters infrastructure IPs
|
||||
- Extracts MAC → +0.50 certainty bump
|
||||
|
||||
3. **DHCP Sniffer** — AF_PACKET raw socket on data interface
|
||||
- Captures DHCP DISCOVER/REQUEST packets
|
||||
- Extracts client MAC (chaddr field)
|
||||
- Extracts hostname (option 12)
|
||||
- MAC → +0.80 certainty bump
|
||||
|
||||
4. **BLE Scanner** — Passive BLE device discovery
|
||||
- Uses `bleak` library if available
|
||||
- Gracefully disables if ImportError
|
||||
- Device address → +0.40 certainty bump
|
||||
|
||||
### Core Logic
|
||||
|
||||
**Signal Fusion** (per person):
|
||||
- Collect all device certainties
|
||||
- Apply exponential decay: `certainty × e^(-0.08 × age_minutes)`
|
||||
- Aggregate: `max + 0.08 × second_max`
|
||||
- Check state machine
|
||||
|
||||
**Presence Anchor** (critical):
|
||||
- Once PRESENT (certainty ≥ 0.40), hold PRESENT for 45 minutes from last signal
|
||||
- Prevents false ABSENT from temporary signal loss (iOS suppression, WiFi roam, power save)
|
||||
- After 45 minutes without signal ≥ 0.40 → ABSENT
|
||||
|
||||
**State Machine**:
|
||||
```
|
||||
UNKNOWN → PRESENT → ABSENT → PRESENT
|
||||
↓ agg_cert ↓
|
||||
└─────────≥ 0.40 45 min no signal
|
||||
```
|
||||
|
||||
**Alerting**:
|
||||
- UNKNOWN → PRESENT: "ARRIVED" alert
|
||||
- PRESENT → ABSENT: "DEPARTED" alert
|
||||
- Via Matrix webhook if configured
|
||||
|
||||
### Database Schema
|
||||
|
||||
**persons** — Named people
|
||||
- id (PK)
|
||||
- name — Person name
|
||||
- notes — Optional metadata
|
||||
|
||||
**devices** — MAC-to-person mapping
|
||||
- mac (PK)
|
||||
- person_id (FK)
|
||||
- label — Device name (iPhone, Laptop, etc.)
|
||||
- added_at — Timestamp
|
||||
|
||||
**signals** — Raw signal observations
|
||||
- id (PK)
|
||||
- mac — Device MAC
|
||||
- signal_type — "probe", "arp", "dhcp", or "ble"
|
||||
- certainty — Aggregated certainty after this signal
|
||||
- ts — Observation timestamp
|
||||
|
||||
**occupancy_log** — State transitions
|
||||
- id (PK)
|
||||
- person_id (FK)
|
||||
- old_status — Previous state
|
||||
- new_status — New state
|
||||
- ts — Transition timestamp
|
||||
|
||||
## HTTP Status Endpoint
|
||||
|
||||
GET http://127.0.0.1:9191/
|
||||
|
||||
Returns JSON:
|
||||
```json
|
||||
{
|
||||
"persons": [
|
||||
{
|
||||
"id": 1,
|
||||
"name": "Alice",
|
||||
"status": "PRESENT",
|
||||
"last_change": 1712761234.5
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Limitations (Phase 1)
|
||||
|
||||
- **No auto-enrollment**: Device-to-person mapping must be set up manually in database
|
||||
- **No mDNS parsing**: Hostnames are not extracted or used for identity
|
||||
- **BLE optional**: Requires `bleak` library; gracefully disabled if missing
|
||||
- **Monitor mode required**: Probe sniffer needs separate WiFi adapter in monitor mode
|
||||
- **Root access needed**: Raw sockets require CAP_NET_ADMIN or root
|
||||
- **Single location**: No multi-room support
|
||||
|
||||
## Files
|
||||
|
||||
- `presence_daemon.py` — Main daemon (1000+ lines)
|
||||
- `presence_schema.sql` — SQLite schema
|
||||
- `install.sh` — Deployment script
|
||||
- `AUDIT_impl.md` — Implementation notes and audit checklist
|
||||
- `README.md` — This file
|
||||
|
||||
## Logging
|
||||
|
||||
Logs to `/var/log/sensor.log` and stdout. Log level: INFO (debug messages only on errors).
|
||||
|
||||
Example log output:
|
||||
```
|
||||
2026-04-10 14:54:32,123 [INFO] Database initialized at /opt/sensor/sensor.db
|
||||
2026-04-10 14:54:32,124 [INFO] Loaded 2 persons from database
|
||||
2026-04-10 14:54:32,125 [INFO] Probe sniffer started on wlan1
|
||||
2026-04-10 14:54:32,126 [INFO] ARP listener started
|
||||
2026-04-10 14:54:32,127 [INFO] DHCP sniffer started on wlan0
|
||||
2026-04-10 14:54:32,128 [INFO] BLE scanner started
|
||||
2026-04-10 14:54:32,129 [INFO] Status server started on http://127.0.0.1:9191
|
||||
2026-04-10 14:54:32,130 [INFO] Sensor daemon started
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
1. Verify systemd service is running:
|
||||
```bash
|
||||
sudo systemctl status sensor
|
||||
```
|
||||
|
||||
2. Check database initialization:
|
||||
```bash
|
||||
sqlite3 /opt/sensor/sensor.db ".tables"
|
||||
```
|
||||
|
||||
3. Query persons:
|
||||
```bash
|
||||
sqlite3 /opt/sensor/sensor.db "SELECT * FROM persons;"
|
||||
```
|
||||
|
||||
4. Query recent signals:
|
||||
```bash
|
||||
sqlite3 /opt/sensor/sensor.db "SELECT * FROM signals ORDER BY ts DESC LIMIT 10;"
|
||||
```
|
||||
|
||||
5. Check status endpoint:
|
||||
```bash
|
||||
curl http://127.0.0.1:9191/
|
||||
```
|
||||
|
||||
6. Check logs:
|
||||
```bash
|
||||
tail -f /var/log/sensor.log
|
||||
```
|
||||
|
||||
## Phase 2 Roadmap
|
||||
|
||||
- Auto-enrollment from mDNS hostnames
|
||||
- BLE MAC signature recognition (Apple Watch, AirPods)
|
||||
- Device quorum (multi-device presence confirmation)
|
||||
- Tentative device linking via co-occurrence windows
|
||||
|
||||
## Phase 3 Roadmap
|
||||
|
||||
- Pattern of life analytics
|
||||
- Arrival/departure time learning
|
||||
- Occupancy forecasting
|
||||
- Multi-location aggregation
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/bin/bash
|
||||
# Install script for presence daemon
|
||||
# Deploys sensor.py and service configuration to /opt/sensor/
|
||||
|
||||
set -e
|
||||
|
||||
echo "Installing presence daemon..."
|
||||
|
||||
# Create sensor directory
|
||||
sudo mkdir -p /opt/sensor
|
||||
sudo chown $USER:$USER /opt/sensor
|
||||
|
||||
# Copy daemon and schema
|
||||
cp presence_daemon.py /opt/sensor/sensor.py
|
||||
cp presence_schema.sql /opt/sensor/
|
||||
chmod +x /opt/sensor/sensor.py
|
||||
|
||||
echo "Created /opt/sensor/sensor.py and schema"
|
||||
|
||||
# Create systemd service
|
||||
sudo tee /etc/systemd/system/sensor.service > /dev/null << 'EOF'
|
||||
[Unit]
|
||||
Description=Network Sensor
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/bin/python3 /opt/sensor/sensor.py
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
Environment=PRESENCE_DB_PATH=/opt/sensor/presence.db
|
||||
# Optional: override default status port (9191)
|
||||
# Environment=PRESENCE_STATUS_PORT=9191
|
||||
# Optional: configure Matrix webhook for alerts
|
||||
# Environment=PRESENCE_MATRIX_WEBHOOK=https://matrix.example.com/hook
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
echo "Created /etc/systemd/system/sensor.service"
|
||||
|
||||
# Reload systemd and enable service
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable sensor
|
||||
sudo systemctl start sensor
|
||||
|
||||
echo "Enabled and started sensor service"
|
||||
echo "Status:"
|
||||
sudo systemctl status sensor --no-pager | head -20
|
||||
@@ -0,0 +1,719 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Presence daemon — Multi-signal WiFi/ARP/DHCP/BLE person presence tracking.
|
||||
Passive sensors with exponential decay and presence anchoring.
|
||||
Zero active probing. No tool fingerprints in output.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import http.server
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import socket
|
||||
import sqlite3
|
||||
import struct
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# --- Config (env-driven) ---
|
||||
MONITOR_IFACE = os.getenv("PRESENCE_MONITOR_IFACE", "wlan1")
|
||||
DATA_IFACE = os.getenv("PRESENCE_DATA_IFACE", "wlan0")
|
||||
DB_PATH = os.getenv("PRESENCE_DB_PATH", "/opt/sensor/presence.db")
|
||||
MATRIX_WEBHOOK = os.getenv("PRESENCE_MATRIX_WEBHOOK", "")
|
||||
STATUS_PORT = int(os.getenv("PRESENCE_STATUS_PORT", "9191"))
|
||||
PRESENCE_THRESHOLD = 0.40
|
||||
ABSENCE_THRESHOLD = 0.10
|
||||
ANCHOR_MINUTES = 45
|
||||
DECAY_LAMBDA = 0.08
|
||||
|
||||
# Signal bump constants
|
||||
PROBE_BUMP = 0.60
|
||||
ARP_BUMP = 0.50
|
||||
DHCP_BUMP = 0.80
|
||||
BLE_BUMP = 0.40
|
||||
|
||||
# Logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
handlers=[
|
||||
logging.FileHandler("/var/log/sensor.log", encoding="utf-8"),
|
||||
logging.StreamHandler(sys.stdout)
|
||||
]
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Top OUI prefixes (inline, no external DB)
|
||||
OUI_DICT = {
|
||||
"88A29E": "Apple",
|
||||
"001A7D": "Apple",
|
||||
"185F3F": "Apple",
|
||||
"A4C3F0": "Apple",
|
||||
"0899D8": "Apple",
|
||||
"004096": "Apple",
|
||||
"00219B": "Apple",
|
||||
"0021E9": "Apple",
|
||||
"005973": "Apple",
|
||||
"006377": "Apple",
|
||||
"0064B9": "Apple",
|
||||
"0084F3": "Apple",
|
||||
"00A04D": "Apple",
|
||||
"00D04B": "Apple",
|
||||
"28879F": "Google",
|
||||
"2887BA": "Google",
|
||||
"5427EB": "Google",
|
||||
"542758": "Google",
|
||||
"341513": "Amazon",
|
||||
"0C47C2": "Amazon",
|
||||
"B827EB": "Raspberry Pi",
|
||||
"2C56DC": "Raspberry Pi",
|
||||
"E45F01": "Raspberry Pi",
|
||||
}
|
||||
|
||||
# Netlink constants
|
||||
NETLINK_ROUTE = 0
|
||||
RTMGRP_NEIGH = 0x4
|
||||
RTM_NEWNEIGH = 28
|
||||
RTM_DELNEIGH = 29
|
||||
NUD_REACHABLE = 0x02
|
||||
NUD_STALE = 0x04
|
||||
NUD_DELAY = 0x08
|
||||
NUD_PROBE = 0x10
|
||||
NDA_DST = 1
|
||||
NDA_LLADDR = 2
|
||||
|
||||
|
||||
@dataclass
|
||||
class DeviceState:
|
||||
"""In-memory device state."""
|
||||
mac: str
|
||||
certainty: float = 0.0
|
||||
last_seen: float = field(default_factory=time.time)
|
||||
last_signal: str = ""
|
||||
person_id: Optional[int] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class PersonState:
|
||||
"""In-memory person state."""
|
||||
person_id: int
|
||||
name: str
|
||||
status: str = "UNKNOWN"
|
||||
last_change: float = field(default_factory=time.time)
|
||||
|
||||
|
||||
# Shared state with locks
|
||||
devices = {}
|
||||
devices_lock = threading.Lock()
|
||||
|
||||
persons = {}
|
||||
persons_lock = threading.Lock()
|
||||
|
||||
infrastructure_ips = set()
|
||||
infrastructure_lock = threading.Lock()
|
||||
|
||||
|
||||
def lookup_oui(mac: str) -> str:
|
||||
"""Look up OUI vendor from MAC (first 3 octets, inline dict)."""
|
||||
if not mac or mac == "00:00:00:00:00:00":
|
||||
return "unknown"
|
||||
oui_prefix = mac.replace(":", "").upper()[:6]
|
||||
return OUI_DICT.get(oui_prefix, "unknown")
|
||||
|
||||
|
||||
def init_db() -> None:
|
||||
"""Initialize SQLite database from schema if not exists."""
|
||||
schema_path = Path(__file__).parent / "presence_schema.sql"
|
||||
if not schema_path.exists():
|
||||
logger.error(f"Schema file not found: {schema_path}")
|
||||
return
|
||||
|
||||
db_path = Path(DB_PATH)
|
||||
db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
conn = sqlite3.connect(DB_PATH, timeout=10)
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
schema = schema_path.read_text()
|
||||
cursor.executescript(schema)
|
||||
conn.commit()
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
conn.commit()
|
||||
logger.info(f"Database initialized at {DB_PATH}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to init database: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def lookup_person(mac: str) -> Optional[int]:
|
||||
"""Look up person_id for a MAC from database."""
|
||||
conn = sqlite3.connect(DB_PATH, timeout=10)
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
result = cursor.execute(
|
||||
"SELECT person_id FROM devices WHERE mac = ?",
|
||||
(mac.lower(),)
|
||||
).fetchone()
|
||||
return result[0] if result else None
|
||||
except Exception as e:
|
||||
logger.debug(f"lookup_person error for {mac}: {e}")
|
||||
return None
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def seed_persons_from_db() -> None:
|
||||
"""Load all persons from database into memory."""
|
||||
conn = sqlite3.connect(DB_PATH, timeout=10)
|
||||
try:
|
||||
conn.row_factory = sqlite3.Row
|
||||
cursor = conn.cursor()
|
||||
rows = cursor.execute("SELECT id, name FROM persons").fetchall()
|
||||
|
||||
with persons_lock:
|
||||
for row in rows:
|
||||
person = PersonState(
|
||||
person_id=row["id"],
|
||||
name=row["name"]
|
||||
)
|
||||
persons[row["id"]] = person
|
||||
logger.info(f"Loaded {len(rows)} persons from database")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to seed persons: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def log_signal(mac: str, signal_type: str, certainty: float) -> None:
|
||||
"""Insert signal into database (non-blocking via queue would be ideal, but inline for simplicity)."""
|
||||
conn = sqlite3.connect(DB_PATH, timeout=10)
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute(
|
||||
"INSERT INTO signals (mac, signal_type, certainty, ts) VALUES (?, ?, ?, ?)",
|
||||
(mac.lower(), signal_type, certainty, time.time())
|
||||
)
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to log signal: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def log_presence_change(person_id: int, old_status: str, new_status: str) -> None:
|
||||
"""Insert occupancy change into database."""
|
||||
conn = sqlite3.connect(DB_PATH, timeout=10)
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute(
|
||||
"INSERT INTO occupancy_log (person_id, old_status, new_status, ts) VALUES (?, ?, ?, ?)",
|
||||
(person_id, old_status, new_status, time.time())
|
||||
)
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to log presence change: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def send_alert(person_name: str, status: str, signal_type: str, certainty: float) -> None:
|
||||
"""Send alert to Matrix webhook if configured."""
|
||||
if not MATRIX_WEBHOOK:
|
||||
logger.info(f"Alert (no webhook): {person_name} — {status} ({signal_type}, certainty: {certainty:.2f})")
|
||||
return
|
||||
|
||||
try:
|
||||
action = "ARRIVED" if status == "PRESENT" else "DEPARTED"
|
||||
message = f"SENSOR: {person_name} — {action} ({signal_type}, certainty: {certainty:.2f})"
|
||||
payload = json.dumps({"text": message})
|
||||
req = urllib.request.Request(
|
||||
MATRIX_WEBHOOK,
|
||||
data=payload.encode("utf-8"),
|
||||
headers={"Content-Type": "application/json"}
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=5) as response:
|
||||
logger.info(f"Alert sent: {message}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to send alert: {e}")
|
||||
|
||||
|
||||
def on_signal(mac: str, signal_type: str, bump: float) -> None:
|
||||
"""
|
||||
Process a signal bump for a device.
|
||||
Decay old certainty, apply bump, update state.
|
||||
"""
|
||||
mac = mac.lower()
|
||||
with devices_lock:
|
||||
dev = devices.get(mac)
|
||||
if dev is None:
|
||||
dev = DeviceState(
|
||||
mac=mac,
|
||||
certainty=0.0,
|
||||
last_seen=time.time(),
|
||||
last_signal="",
|
||||
person_id=lookup_person(mac)
|
||||
)
|
||||
devices[mac] = dev
|
||||
|
||||
# Apply exponential decay since last signal
|
||||
elapsed_minutes = (time.time() - dev.last_seen) / 60.0
|
||||
dev.certainty = dev.certainty * math.exp(-DECAY_LAMBDA * elapsed_minutes)
|
||||
|
||||
# Apply bump
|
||||
dev.certainty = min(1.0, dev.certainty + bump)
|
||||
dev.last_seen = time.time()
|
||||
dev.last_signal = signal_type
|
||||
|
||||
log_signal(mac, signal_type, dev.certainty)
|
||||
update_person_state(dev.person_id)
|
||||
|
||||
|
||||
def update_person_state(person_id: Optional[int]) -> None:
|
||||
"""
|
||||
Compute aggregate certainty for a person, update state machine.
|
||||
Fire alerts on state transitions.
|
||||
"""
|
||||
if person_id is None:
|
||||
return
|
||||
|
||||
# Snapshot device state under lock
|
||||
with devices_lock:
|
||||
# Get all devices for this person
|
||||
devices_for_person = [d for d in devices.values() if d.person_id == person_id]
|
||||
if not devices_for_person:
|
||||
return
|
||||
|
||||
# Compute aggregate certainty (max + 0.08 * second_max)
|
||||
certainties = sorted([d.certainty for d in devices_for_person], reverse=True)
|
||||
if len(certainties) >= 2:
|
||||
agg_certainty = certainties[0] + 0.08 * certainties[1]
|
||||
else:
|
||||
agg_certainty = certainties[0]
|
||||
|
||||
# Get max last_seen time
|
||||
max_last_seen = max(d.last_seen for d in devices_for_person)
|
||||
|
||||
# Determine new status based on snapshot
|
||||
current_time = time.time()
|
||||
elapsed_since_signal = (current_time - max_last_seen) / 60.0
|
||||
|
||||
with persons_lock:
|
||||
person = persons.get(person_id)
|
||||
if person is None:
|
||||
return
|
||||
|
||||
old_status = person.status
|
||||
new_status = old_status
|
||||
|
||||
# State machine logic
|
||||
if person.status == "UNKNOWN":
|
||||
if agg_certainty >= PRESENCE_THRESHOLD:
|
||||
new_status = "PRESENT"
|
||||
elif person.status == "PRESENT":
|
||||
# Check presence anchor
|
||||
if elapsed_since_signal >= ANCHOR_MINUTES:
|
||||
new_status = "ABSENT"
|
||||
elif person.status == "ABSENT":
|
||||
if agg_certainty >= PRESENCE_THRESHOLD:
|
||||
new_status = "PRESENT"
|
||||
|
||||
# Apply state transition if changed
|
||||
if new_status != old_status:
|
||||
person.status = new_status
|
||||
person.last_change = current_time
|
||||
log_presence_change(person_id, old_status, new_status)
|
||||
|
||||
# Send alert outside of lock
|
||||
if new_status != old_status:
|
||||
if new_status == "PRESENT":
|
||||
send_alert(person.name, "PRESENT", "signal", agg_certainty)
|
||||
else:
|
||||
send_alert(person.name, "ABSENT", "timeout", agg_certainty)
|
||||
|
||||
|
||||
def probe_sniffer(iface: str) -> None:
|
||||
"""
|
||||
WiFi probe request sniffer.
|
||||
Raw 802.11 frame capture on monitor mode interface.
|
||||
Extracts source MAC from probe requests.
|
||||
"""
|
||||
try:
|
||||
s = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(0x0003))
|
||||
s.bind((iface, 0))
|
||||
logger.info(f"Probe sniffer started on {iface}")
|
||||
|
||||
while True:
|
||||
try:
|
||||
data, _ = s.recvfrom(65535)
|
||||
if len(data) < 24:
|
||||
continue
|
||||
|
||||
# 802.11 frame: FC (2) + duration (2) + dest (6) + src (6) + BSSID (6) = 22 bytes min
|
||||
fc = struct.unpack("<H", data[0:2])[0]
|
||||
frame_type = (fc >> 2) & 0x3
|
||||
frame_subtype = (fc >> 4) & 0xf
|
||||
|
||||
# Type 0 = management, Subtype 4 = probe request
|
||||
if frame_type == 0 and frame_subtype == 4:
|
||||
if len(data) >= 16:
|
||||
# Source MAC at bytes 10-16
|
||||
mac_bytes = data[10:16]
|
||||
mac = ':'.join(f'{b:02x}' for b in mac_bytes)
|
||||
if mac != "ff:ff:ff:ff:ff:ff" and mac != "00:00:00:00:00:00":
|
||||
on_signal(mac, "probe", PROBE_BUMP)
|
||||
except Exception as e:
|
||||
logger.debug(f"Probe sniffer error: {e}")
|
||||
time.sleep(0.1)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start probe sniffer: {e}")
|
||||
|
||||
|
||||
def arp_listener() -> None:
|
||||
"""
|
||||
ARP listener via netlink RTM_NEWNEIGH.
|
||||
Kernel pushes neighbor discovery events.
|
||||
"""
|
||||
try:
|
||||
s = socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, NETLINK_ROUTE)
|
||||
s.bind((os.getpid(), RTMGRP_NEIGH))
|
||||
logger.info("ARP listener started")
|
||||
|
||||
while True:
|
||||
try:
|
||||
data = s.recv(65535)
|
||||
parse_netlink(data)
|
||||
except Exception as e:
|
||||
logger.debug(f"ARP listener error: {e}")
|
||||
time.sleep(0.1)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start ARP listener: {e}")
|
||||
|
||||
|
||||
def parse_netlink(data: bytes) -> None:
|
||||
"""Parse netlink messages for neighbor events."""
|
||||
try:
|
||||
offset = 0
|
||||
while offset < len(data):
|
||||
if offset + 16 > len(data):
|
||||
break
|
||||
|
||||
nlmsg_len, nlmsg_type, _, _, _ = struct.unpack_from('=IHHII', data, offset)
|
||||
if nlmsg_len < 16:
|
||||
break
|
||||
|
||||
# Verify complete message is available before extracting payload
|
||||
if offset + nlmsg_len > len(data):
|
||||
return
|
||||
|
||||
payload = data[offset + 16:offset + nlmsg_len]
|
||||
|
||||
if nlmsg_type in (RTM_NEWNEIGH, RTM_DELNEIGH):
|
||||
parse_ndmsg(payload, nlmsg_type)
|
||||
|
||||
offset += (nlmsg_len + 3) & ~3
|
||||
except Exception as e:
|
||||
logger.debug(f"Netlink parse error: {e}")
|
||||
|
||||
|
||||
def parse_ndmsg(data: bytes, msg_type: int) -> None:
|
||||
"""Parse ndmsg (neighbor discovery message)."""
|
||||
try:
|
||||
if len(data) < 12:
|
||||
return
|
||||
|
||||
state = struct.unpack_from('=H', data, 8)[0]
|
||||
|
||||
mac = None
|
||||
ip = None
|
||||
offset = 12
|
||||
|
||||
while offset + 4 <= len(data):
|
||||
rta_len, rta_type = struct.unpack_from('=HH', data, offset)
|
||||
if rta_len < 4:
|
||||
break
|
||||
|
||||
# Verify complete RTA attribute is available before extracting
|
||||
if offset + rta_len > len(data):
|
||||
return
|
||||
|
||||
val = data[offset + 4:offset + rta_len]
|
||||
|
||||
if rta_type == NDA_LLADDR and len(val) == 6:
|
||||
mac = ':'.join(f'{b:02x}' for b in val)
|
||||
elif rta_type == NDA_DST:
|
||||
if len(val) == 4:
|
||||
ip = socket.inet_ntoa(val)
|
||||
|
||||
offset += (rta_len + 3) & ~3
|
||||
|
||||
if not mac or mac in ('00:00:00:00:00:00', 'ff:ff:ff:ff:ff:ff'):
|
||||
return
|
||||
|
||||
# Ignore infrastructure IPs
|
||||
with infrastructure_lock:
|
||||
if ip and ip in infrastructure_ips:
|
||||
return
|
||||
|
||||
if msg_type == RTM_NEWNEIGH and (state & (NUD_REACHABLE | NUD_STALE | NUD_DELAY | NUD_PROBE)):
|
||||
on_signal(mac, "arp", ARP_BUMP)
|
||||
except Exception as e:
|
||||
logger.debug(f"ndmsg parse error: {e}")
|
||||
|
||||
|
||||
def dhcp_sniffer(iface: str) -> None:
|
||||
"""
|
||||
DHCP sniffer via AF_PACKET raw socket.
|
||||
Captures DHCP traffic passively, extracts client MACs.
|
||||
"""
|
||||
try:
|
||||
s = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(0x0003))
|
||||
s.bind((iface, 0))
|
||||
logger.info(f"DHCP sniffer started on {iface}")
|
||||
|
||||
while True:
|
||||
try:
|
||||
data, _ = s.recvfrom(65535)
|
||||
parse_dhcp(data)
|
||||
except Exception as e:
|
||||
logger.debug(f"DHCP sniffer error: {e}")
|
||||
time.sleep(0.1)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start DHCP sniffer: {e}")
|
||||
|
||||
|
||||
def parse_dhcp(data: bytes) -> None:
|
||||
"""Parse DHCP packet and extract client MAC."""
|
||||
try:
|
||||
if len(data) < 14:
|
||||
return
|
||||
|
||||
# Ethernet frame
|
||||
eth_type = struct.unpack('!H', data[12:14])[0]
|
||||
if eth_type != 0x0800: # not IPv4
|
||||
return
|
||||
|
||||
# IP header
|
||||
if len(data) < 23:
|
||||
return
|
||||
proto = data[23]
|
||||
if proto != 17: # not UDP
|
||||
return
|
||||
|
||||
# UDP ports
|
||||
if len(data) < 38:
|
||||
return
|
||||
src_port = struct.unpack('!H', data[34:36])[0]
|
||||
dst_port = struct.unpack('!H', data[36:38])[0]
|
||||
if not (src_port in (67, 68) and dst_port in (67, 68)):
|
||||
return
|
||||
|
||||
# DHCP payload (starts at byte 42)
|
||||
if len(data) < 42:
|
||||
return
|
||||
dhcp = data[42:]
|
||||
if len(dhcp) < 236:
|
||||
return
|
||||
|
||||
# MAC address (chaddr at offset 28, 6 bytes)
|
||||
if len(dhcp) < 34:
|
||||
return
|
||||
mac_bytes = dhcp[28:34]
|
||||
mac = ':'.join(f'{b:02x}' for b in mac_bytes)
|
||||
if mac in ('00:00:00:00:00:00', 'ff:ff:ff:ff:ff:ff'):
|
||||
return
|
||||
|
||||
# Parse DHCP options (start at byte 240)
|
||||
if len(dhcp) < 240:
|
||||
return
|
||||
|
||||
msg_type = None
|
||||
i = 240
|
||||
while i < len(dhcp):
|
||||
opt = dhcp[i]
|
||||
if opt == 255:
|
||||
break
|
||||
if opt == 0:
|
||||
i += 1
|
||||
continue
|
||||
if i + 1 >= len(dhcp):
|
||||
break
|
||||
|
||||
length = dhcp[i + 1]
|
||||
if i + 2 + length > len(dhcp):
|
||||
break
|
||||
|
||||
val = dhcp[i + 2:i + 2 + length]
|
||||
|
||||
if opt == 53 and length == 1: # DHCP message type
|
||||
msg_type = val[0]
|
||||
|
||||
i += 2 + length
|
||||
|
||||
# DISCOVER or REQUEST indicates presence
|
||||
if msg_type in (1, 3):
|
||||
on_signal(mac, "dhcp", DHCP_BUMP)
|
||||
except Exception as e:
|
||||
logger.debug(f"DHCP parse error: {e}")
|
||||
|
||||
|
||||
def ble_scanner() -> None:
|
||||
"""
|
||||
BLE passive scanner.
|
||||
Gracefully degrades if bleak is not available.
|
||||
"""
|
||||
try:
|
||||
from bleak import BleakScanner
|
||||
except ImportError:
|
||||
logger.warning("bleak not available; BLE scanning disabled")
|
||||
return
|
||||
|
||||
async def run_ble_scan() -> None:
|
||||
"""Run BLE scan in asyncio loop."""
|
||||
def on_ble_detect(device, adv_data):
|
||||
"""Callback for BLE device detection."""
|
||||
if device and device.address:
|
||||
on_signal(device.address.lower(), "ble", BLE_BUMP)
|
||||
|
||||
try:
|
||||
async with BleakScanner(
|
||||
detection_callback=on_ble_detect,
|
||||
scanning_mode="passive"
|
||||
):
|
||||
await asyncio.sleep(float('inf'))
|
||||
except Exception as e:
|
||||
logger.error(f"BLE scanner error: {e}")
|
||||
|
||||
try:
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
loop.run_until_complete(run_ble_scan())
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start BLE scanner: {e}")
|
||||
|
||||
|
||||
def decay_loop() -> None:
|
||||
"""
|
||||
Background thread: apply decay to all devices every 60 seconds.
|
||||
Prune devices not seen in 120+ minutes.
|
||||
"""
|
||||
while True:
|
||||
try:
|
||||
time.sleep(60)
|
||||
|
||||
with devices_lock:
|
||||
now = time.time()
|
||||
to_prune = []
|
||||
|
||||
for mac, dev in devices.items():
|
||||
elapsed_minutes = (now - dev.last_seen) / 60.0
|
||||
|
||||
# Apply decay
|
||||
dev.certainty = dev.certainty * math.exp(-DECAY_LAMBDA * elapsed_minutes)
|
||||
|
||||
# Mark for pruning if not seen in 120+ minutes
|
||||
if elapsed_minutes >= 120:
|
||||
to_prune.append(mac)
|
||||
|
||||
# Prune old devices
|
||||
for mac in to_prune:
|
||||
del devices[mac]
|
||||
logger.debug(f"Pruned device {mac}")
|
||||
|
||||
# Update all person states due to decay
|
||||
for person_id in set(d.person_id for d in devices.values() if d.person_id):
|
||||
update_person_state(person_id)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Decay loop error: {e}")
|
||||
|
||||
|
||||
def status_handler(request, client_address, server):
|
||||
"""HTTP request handler for status endpoint."""
|
||||
class StatusHandler(http.server.BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
if self.path == "/":
|
||||
try:
|
||||
with persons_lock:
|
||||
person_list = [
|
||||
{
|
||||
"id": p.person_id,
|
||||
"name": p.name,
|
||||
"status": p.status,
|
||||
"last_change": p.last_change
|
||||
}
|
||||
for p in persons.values()
|
||||
]
|
||||
response = json.dumps({"persons": person_list})
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(response.encode("utf-8"))
|
||||
except Exception as e:
|
||||
logger.error(f"Status handler error: {e}")
|
||||
self.send_response(500)
|
||||
self.end_headers()
|
||||
else:
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, format, *args):
|
||||
"""Suppress HTTP server logs."""
|
||||
pass
|
||||
|
||||
return StatusHandler(request, client_address, server)
|
||||
|
||||
|
||||
def status_server() -> None:
|
||||
"""Start HTTP status server on 127.0.0.1:STATUS_PORT."""
|
||||
try:
|
||||
server = http.server.HTTPServer(
|
||||
("127.0.0.1", STATUS_PORT),
|
||||
status_handler
|
||||
)
|
||||
logger.info(f"Status server started on http://127.0.0.1:{STATUS_PORT}")
|
||||
server.serve_forever()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start status server: {e}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Initialize and run all sensor threads."""
|
||||
init_db()
|
||||
seed_persons_from_db()
|
||||
|
||||
threads = [
|
||||
threading.Thread(target=probe_sniffer, args=(MONITOR_IFACE,), daemon=True, name="probe-sniffer"),
|
||||
threading.Thread(target=arp_listener, daemon=True, name="arp-listener"),
|
||||
threading.Thread(target=dhcp_sniffer, args=(DATA_IFACE,), daemon=True, name="dhcp-sniffer"),
|
||||
threading.Thread(target=ble_scanner, daemon=True, name="ble-scanner"),
|
||||
threading.Thread(target=decay_loop, daemon=True, name="decay-loop"),
|
||||
threading.Thread(target=status_server, daemon=True, name="status-server"),
|
||||
]
|
||||
|
||||
for t in threads:
|
||||
t.start()
|
||||
|
||||
logger.info("Sensor daemon started")
|
||||
|
||||
# Handle shutdown gracefully
|
||||
import signal
|
||||
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
|
||||
signal.signal(signal.SIGINT, lambda *_: sys.exit(0))
|
||||
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,33 @@
|
||||
CREATE TABLE IF NOT EXISTS persons (
|
||||
id INTEGER PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
notes TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS devices (
|
||||
mac TEXT PRIMARY KEY,
|
||||
person_id INTEGER REFERENCES persons(id),
|
||||
label TEXT,
|
||||
added_at REAL DEFAULT (unixepoch())
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS signals (
|
||||
id INTEGER PRIMARY KEY,
|
||||
mac TEXT NOT NULL,
|
||||
signal_type TEXT NOT NULL,
|
||||
certainty REAL NOT NULL,
|
||||
ts REAL DEFAULT (unixepoch())
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS occupancy_log (
|
||||
id INTEGER PRIMARY KEY,
|
||||
person_id INTEGER REFERENCES persons(id),
|
||||
old_status TEXT,
|
||||
new_status TEXT NOT NULL,
|
||||
ts REAL DEFAULT (unixepoch())
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_signals_mac ON signals(mac);
|
||||
CREATE INDEX IF NOT EXISTS idx_signals_ts ON signals(ts);
|
||||
CREATE INDEX IF NOT EXISTS idx_occupancy_person ON occupancy_log(person_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_devices_person ON devices(person_id);
|
||||
Reference in New Issue
Block a user