The phone/wearable gate used a manual OUI check that missed privately-addressed
MACs (iOS 14+/Android 10+ randomize per network). Any phone not pre-configured
in device_labels was silently enrolled with no alert, causing ARRIVED/OCCUPIED
to never fire. Replacing the gate with is_personal_device() catches OUI, labels,
and LAA bit uniformly.
ARP scan interval drops from 30s to 15s so worst-case arrival detection is 15s.
Watchdog timeout and departure debounce were both 900s — stacked,
worst-case VACANT alert was 30 minutes after leaving. With active ARP
scanning every 30s we can confirm absence quickly, so both timers drop
to 180s. Override via NET_ALERTER_WATCHDOG_TIMEOUT and
NET_ALERTER_DEPARTURE_DEBOUNCE env vars.
Non-phone OUIs (IoT, thermostats, TVs, etc.) are silently enrolled
for presence tracking but never trigger Matrix alerts. Only devices
matching MOBILE_DEVICE_OUIS or with an explicit label in device_labels
produce alerts.
Passive ARP sniffing fails on modern WiFi because the AP intercepts
ARP broadcasts and responds on behalf of clients. Active scanning
sends who-has to all 254 /24 addresses and collects replies directly,
making presence detection reliable regardless of AP configuration.
Also adds monitor mode sniffer thread that activates automatically if
a second WiFi interface is present (e.g. USB dongle). Reads raw 802.11
frames, bypassing the AP stack entirely for station-transmitted data
and management frames.
When systemd sends SIGTERM, asyncio cancels the scan task before
BleakScanner.__aexit__ runs. BlueZ already stopped discovery by then,
so stop() throws 'No discovery started'. This was crashing the service
and triggering a 22-restart loop before systemd settled. Catch the
specific error string and treat it as a debug-level no-op.
is_labeled compared raw src_mac against normalized device_labels keys,
always evaluating False for labeled devices. ARP replies (opcode=2) from
personal MACs were never trusted, so on_arrival only fired on ARP
requests (opcode=1) which are rare. Labeled devices now correctly pass
the opcode=2 gate, triggering arrival alerts with their display name.
ble_alerter.py: probe Apple BLE devices via GATT characteristic 0x2A00
to discover device names (iPhone, Apple TV, etc). Names persist to
ble_names.json with last_seen timestamps. GATT_PROBE=1 enables active
probing; without it, Apple device last_seen timestamps still tracked.
net_alerter.py: on WiFi device arrival, read ble_names.json and log
any Apple BLE devices seen within the last 60s as correlation candidates.
Gives operator visibility into which BLE MACs co-arrived with a WiFi device.
Shared state file: /opt/net_alerter/ble_names.json
iOS sends 'My iPhone' via DHCP option 12 regardless of the actual device
name set on the phone. Operator-set labels are authoritative and should
not be overridden by privacy-obscured DHCP hostnames.
DHCP triggers on_arrival immediately but iPhone sends mDNS frames
milliseconds later. Without the delay the alert fires with no hostname
and falls back to the device label. Timer reads from known_devices at
fire time so mDNS-updated hostname is used.
Label in device_labels is for tracking/identification. When mDNS reveals
the real device name, use it. Label serves as fallback when no hostname
has been discovered yet.
_extract_mdns_hostname scans A records in mDNS payload before on_arrival fires,
so the ARRIVED alert shows the device's actual name instead of the IP.
_parse_mdns_for_names also now updates known_devices hostname on subsequent frames,
so departure alerts get the real name even if the first frame had no A record.
Operator needs ARRIVED for any phone or wearable on site, including
unknown devices. Silent seeding only applies to infrastructure and
non-personal devices.
seed_from_arp_cache was adding labeled personal devices to known_devices at startup,
causing all subsequent arrivals to be ignored (already_known=True). Now skips them
so on_arrival fires normally when they rejoin.
ARP opcode=2 (replies) are now treated as genuine signals for explicitly labeled devices.
iPhones with privacy MACs frequently emit only opcode=2 after initial association — the
AP proxy forgery concern doesn't apply to devices the operator has explicitly named.
Phone (c6:37:b0:b2:07:30) was cycling through arrivals/departures faster
than the 30-min churn window threshold, causing _check_churn to add its
IP to infrastructure_ips. All subsequent on_arrival calls were silently
dropped by the infrastructure IP check, so no ARRIVED alerts fired.
Two fixes:
- _check_churn: skip churn suppression entirely for MACs in device_labels
or personal_devices — labeled devices are never infrastructure
- on_arrival already-known path: upgrade ip from '0.0.0.0'/'unknown'
(ARP probe placeholder) to real IP when netlink delivers the actual address
Travel routers and other LAA-bit devices get misclassified as personal phones
because the LAA heuristic fires on any MAC with bit 1 set (b6:cc is 0xb6).
Add infrastructure_macs set checked before the LAA heuristic in is_personal_device().
NET_ALERTER_INFRA_MACS env var (comma-separated MACs) populates the set at startup.
b6:cc was mislabeled as 'Dane iPhone' causing it to be treated as a
personal device. Removing the label fixes the problem without needing
any Tailscale-specific code. Keep mDNS and ARP arrival fixes.
tailscale status --json uses CurAddr for the active direct endpoint,
not Endpoints (which is always null/absent). Without this fix the
Tailscale exclusion set was always empty.
Tailscale peers (travel router b6:cc) were kept perpetually REACHABLE in the
kernel ARP cache by OPi's WireGuard keepalives, refreshing last_seen and
blocking departure alerts. seed_tailscale_peers() excludes their MACs.
mDNS was only refreshing the watchdog timer for already-known devices. Apple
devices send mDNS on WiFi join as the primary signal — now calls on_arrival()
so first-seen mDNS triggers ARRIVED alert and occupancy update.
ARP opcode 1 for personal devices never called on_arrival(), so devices that
joined without DHCP (c6:37 My iPhone at 01:25) got no alert. Now fires
on_arrival() for personal devices not yet in known_devices.
Repeated watchdog calls for a device already in departing=True state
were feeding the churn suppressor, which misclassified the iPhone's IP
as infrastructure after 3 calls within 30 min. This blocked all
subsequent departure processing and caused a 31-min alert delay.
Fix: gate the on_departure() call on a should_depart flag set only when
the device is known and not already departing.
Watchdog-triggered departures set departing=True after 15 min of silence.
If the device returned during the 15-min debounce window, on_arrival()
treated it as a rapid flap and suppressed the ARRIVED alert — even though
the device had been genuinely absent for 15+ minutes.
Store depart_initiated timestamp when setting departing=True. On re-arrival,
check the gap: < 120s = true rapid flap (transient Netlink/DHCP event),
suppress as before. >= 120s = real absence confirmed by watchdog, pop the
device from known_devices and let it fall through to the new-device path
so the ARRIVED alert fires.
Also removes debug traceback instrumentation left in _update_last_seen
from the prior investigation session.
Without this, last_seen is empty after every restart and the watchdog
skips all devices (ts is None → continue). Devices that are genuinely
active refresh last_seen via ARP requests or mDNS. AP proxy ghosts only
send ARP replies (now filtered), so they go silent and the watchdog
fires departure after WATCHDOG_TIMEOUT_SEC.
AP ARP proxy sends REPLY frames with the disconnected client's MAC as
the Ethernet source. Treating those as genuine device signals kept
last_seen fresh, preventing the watchdog from detecting departure.
Only ARP REQUEST (opcode 1) frames originate from the device itself.
Skip _update_last_seen() for opcode 2 (REPLY) frames.
ARP proxy on WiFi APs causes the kernel neighbor table to show devices
as REACHABLE even after they disconnect. This made last_seen refresh
continuously from on_arrival() (netlink path), preventing the watchdog
from ever detecting departure.
Two fixes:
1. Remove _update_last_seen() from on_arrival() — netlink events are
not reliable as a genuine-traffic source on ARP-proxy networks.
last_seen is now only updated by the raw packet capture path (ARP
requests, mDNS, DHCP) where src_mac must be the device itself.
2. Watchdog now covers all is_personal_device() entries in known_devices,
not just the personal_devices enrollment set. device_labels entries
(like named phones) were previously unmonitored by the watchdog.
Kernel fires RTM_NEWNEIGH for NUD_STALE, NUD_DELAY, and NUD_PROBE states
as it probes a potentially-departed device. Each probe was calling on_arrival()
which refreshed last_seen, preventing the watchdog from ever detecting departure.
Only NUD_REACHABLE confirms a device is actually on the network.
Garmin and Fitbit wearables use globally-assigned OUI MACs (no private addressing)
so the LAA bit check misses them. Added their OUIs to MOBILE_DEVICE_OUIS.
Consolidated all three detection signals into is_personal_device():
1. Explicit config (device_labels, personal_devices)
2. OUI table (phones + Garmin/Fitbit)
3. LAA bit (iOS 14+/Android 10+ private MACs)
Occupancy _is_personal now delegates to is_personal_device instead of duplicating logic.
Modern phones use private/random MACs (iOS 14+, Android 10+) which
always have the locally-administered bit set (bit 1 of first octet).
TVs, gateways, and smart home devices use globally-assigned OUI MACs
where this bit is clear.
No configuration needed — any LAA MAC counts toward occupancy.
Explicit device_labels/personal_devices still work as fallback.
Previously counted every non-infrastructure device toward OCCUPIED state,
causing TVs and other stationary equipment to falsely indicate occupancy.
Now only MACs in device_labels or personal_devices count. Everything else
is observed but ignored for presence purposes.
device_labels keys are normalized (no colons, uppercase) but known_devices keys
are colon-lowercase. device_labels.get(mac) always returned None.
Also extend label display to arrival and departure alerts — if a MAC has a
configured label, show that instead of vendor/hostname.
NET_ALERTER_DEVICE_LABELS=MAC=Name,MAC=Name maps known devices to
friendly names. OCCUPIED alerts now show which devices are present.
Falls back to vendor string then MAC tail if no label configured.
Previously the service would suppress the startup OCCUPIED alert, meaning
if phones were already present when the service started, no Matrix alert
ever fired until a departure+return cycle. Users never received confirmation
the sensor was working.
Changes:
1. mDNS as separate signal category: Map 'mdns' signal_type to its own signal_category (not grouped with 'wifi'). Enrollment now triggers when EITHER (a) len(signal_types) >= 2 OR (b) 'mdns' in signal_types.
2. Active probes on device discovery: When new identity is created, fire non-blocking probes to collect additional signals. Uses arping or ping (no new deps) to probe IP if available. Runs in daemon thread to avoid stalling correlation loop.
This allows devices announcing via mDNS to be enrolled immediately without waiting for a second signal type, while still collecting additional signals for richer device profiles.
Strategy 0 now checks if a MAC already exists in any identity's wifi_macs/ble_macs
sets before creating a new record. Prevents duplicate identity creation on repeated
mDNS/ARP signals.
Strategy 3 now conditionally updates existing records instead of unconditionally
overwriting them. Preserves signal_count and signal_types accumulation needed for
auto-enrollment (2+ signal types).
Fixes issue where devices seen only via mDNS/ARP never reached enrollment threshold.
Wrap BleakScanner context manager in try-except to detect adapter
initialization errors. If 'No Bluetooth adapters found' or similar, log
warning and set shutdown event to trigger graceful exit with code 0.
Prevents systemd restart loop on hardware without BT adapter (e.g. OPi).
P1 Fixes:
- Signal count enrollment logic: Changed from broken signal_count increment to tracking distinct signal types (BLE vs WiFi) using a set. Device enrolls when len(signal_types) >= 2, ensuring multi-source correlation.
- DNS mDNS pointer endianness: Added bounds check to prevent out-of-bounds reads when following DNS compression pointers. Checks pointer_offset < offset and pointer_offset < len(payload) before recursing.
- Nested RLock fragility: Refactored enrollment callback to not acquire lock (caller _ingest_signal holds it). Renamed _on_device_enrolled() to _fire_enrollment_callback() and removed lock acquisition.
P2 Fixes:
- BLE Handoff parsing: Implemented full HCI packet parsing to extract Apple Company ID (0x004C), Handoff message type (0x0C), and sequence number (bytes 4-5, big-endian). Calls _ingest_signal() with handoff_seq parameter.
- DNS record count overflow: Capped total_records at 1000 to prevent unbounded loop DoS on crafted mDNS packets.
- device_store unbounded growth: Added simple eviction when store exceeds 500 entries - evicts 100 oldest by first_seen timestamp. No LRU needed for MVP.
All 40 existing tests continue to pass.
Track devices across MAC rotations using BLE Handoff sequence numbers
and DHCP Option 55 fingerprinting. Auto-enroll personal devices when
seen on 2+ independent signal types. Handle iOS MAC rotation silently.
Confidence-based departure only fires for enrolled devices.
Changes:
- New device_store dict for stable device identity tracking (keyed by
handoff seq anchor or DHCP fingerprint hash)
- _dhcp_fingerprint_hash() computes stable DHCP Option 55 fingerprint
- _create_identity_record() initializes device records
- _on_device_enrolled() triggers silent enrollment when signal_count >= 2
- _correlate_or_create_identity() finds or creates identities via BLE seq
or DHCP fingerprint, handles MAC rotation detection
- _ingest_signal() ingests signals from multiple sources (BLE, DHCP, ARP,
mDNS) and triggers enrollment when criteria met
- ble_sniffer() thread for Apple Continuity Protocol parsing (HCI socket +
hcidump fallback); graceful failure on missing hardware
- Modified parse_frame() to extract DHCP Option 55 and ingest signals for
ARP, mDNS, DHCP packets
- Updated main() to start BLE sniffer thread and log device_store stats
Implement passive observation tracking for iOS devices:
- Add last_seen dict and lock for tracking passive frame observations
- Parse ARP frames (ethertype 0x0806) for MAC observation
- Parse mDNS frames (UDP port 5353) for device name auto-discovery
- Rename parse_dhcp to parse_frame to reflect broader scope
- Add personal_names set for device name matching
- Parse NET_ALERTER_PERSONAL_NAMES env var (comma-separated device names)
- Add mDNS DNS message parsing to extract PTR/SRV/A record names
- Auto-add MACs to personal_devices when mDNS name matches personal_names
- Call _update_last_seen on all frame types (ARP, mDNS, DHCP)
- Add personal_watchdog thread to fire on_departure after WATCHDOG_TIMEOUT_SEC
- Add NET_ALERTER_WATCHDOG_TIMEOUT env config (default 900s)
- Fix load_personal_macs_from_env logging to show counts instead of MACs
Changes are minimal and focused:
- No socket changes (existing AF_PACKET socket already sees all frames)
- No new dependencies (stdlib only)
- Deadlock prevention (watchdog does not hold locks when calling on_departure)
- Backward compatible (personal_names optional, watchdog is passive)
on_departure() holds known_lock when calling _check_churn(). When churn
threshold is hit, the nested `with known_lock:` inside _check_churn tried
to reacquire an already-held non-reentrant lock → permanent deadlock.
This killed the production process on the OPi Zero 3 after ~15 hours of
runtime on 2026-04-12 at ~02:12 UTC. infrastructure_ips.add() is safe
without the inner lock since the caller already holds known_lock.
Also: reset _churn_tracker between tests to prevent cross-test state
accumulation that triggered the churn threshold in test isolation.
Fix#605: Remove MATRIX_HOMESERVER, MATRIX_ACCESS_TOKEN, MATRIX_ROOM_ID from module-level globals.
Credentials are now read at call time in send_alert() via _read_env_value() to minimize
credential lifetime in process memory.
Fix#606: Update deploy.sh to use Infisical creds CLI instead of .secrets file.
Fetches NET_ALERTER_PASSWORD from matrix folder in Infisical vault, then generates
access token at deploy time.
All 16 net_alerter tests pass when run individually or in groups.
Two P1 bugs fixed:
1. Timer cancel deadlock: timer.cancel() was called while holding locks, which would deadlock if the callback was running/paused waiting for those same locks. Fixed by extracting timers outside of all locks before calling cancel().
2. MAC validation: _normalize_mac() now validates format with regex and returns empty string for invalid MACs. Callers (load_personal_macs_from_env, is_personal_device) skip empty results and log warnings.
All 16 tests pass.
- P1-1: Fix lock ordering deadlock by establishing canonical order (known_lock → departure_lock → occupancy_lock). Refactor on_arrival() to acquire locks in correct order. Fix send_departure_alert() callback to hold occupancy_lock while updating state.
- P1-2: Add _normalize_mac() helper to strip colons/hyphens, apply to load_personal_macs_from_env() and is_personal_device() for consistent MAC comparison.
- P1-3: Remove occupancy_lock acquisition from _update_occupancy_state_unlocked() - callers must hold lock. Update all callers to acquire occupancy_lock before calling.
All individual tests pass. Fixes allow on_arrival() and timer callbacks to execute without deadlock.
- Add MOBILE_DEVICE_OUIS set with 55+ OUI prefixes for personal devices
- Add is_personal_device() function to detect personal devices by OUI or manual config
- Add load_personal_macs_from_env() to parse NET_ALERTER_PERSONAL_MACS
- Add location_occupancy state tracking (VACANT/OCCUPIED/UNKNOWN)
- Add update_occupancy_state() function with thread-safe alerts on transitions
- Add _update_occupancy_state_unlocked() helper to prevent deadlock
- Fix deadlock in on_arrival() by using unlocked version when holding known_lock
- Update test suite with 5 new occupancy tests
- Fix test assertions to exclude occupancy alerts for device arrival tests
Tests now call cleanup_flap_state() at the start of each test to properly
reset the interface flap detection state (_flap_window, _interface_recovering,
_recovery_timer) between tests, preventing state carryover failures.
Cobra