Files
hack-house/cmd_chat/sor/watchdog.py
T
leetcrypt 6bae9a176d DRAFT(next-exp): container-house pools + connectivity failsafe scaffolds
Defensive-measurement instrument scaffolds for the re-staged experiment, not
wired into the sealed pipeline:
- house_pool.py: distinct isolated container pools per house (own net + keys),
  deterministic seed-derived node ids; refuses engine=="local" (containment).
- watchdog.py: pre-registered HALT / PAUSE-RESUME failsafe for load-bearing
  node pools; deterministic, probe injected (no external target).

Both are DRAFT for a fresh pre-registration; neither touches the sealed battery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-22 23:18:16 -07:00

113 lines
4.1 KiB
Python

"""Connectivity failsafe for load-bearing node pools — DRAFT (next experiment only).
Purpose (see prereg NEXT-EXPERIMENT-DRAFT §2): when node pools are *load-bearing* for
the measured data (Track B/D), a run-control watchdog periodically probes every required
pool and, on an outage, fires a **pre-registered** policy — chosen in advance and echoed
into the manifest, never decided after seeing data.
Policies:
* HALT (default, integrity-first): abort the run, mark the cell ``inconclusive`` (a
pre-committed outcome, so there is no optional-stopping bias), seal what exists.
* PAUSE_RESUME (only when determinism is provable): freeze the churn clock + seed
stream, wait (bounded) for the pool to return, then resume from the exact frozen
state. Legitimate only because the seed formula makes the resumed sequence
bit-identical.
Why this did NOT apply to the sealed study: there, the phones were non-forwarding, so no
measured data depended on connectivity. Here the pools ARE in the data path, so the
watchdog protects real integrity.
Self-contained + deterministic: the reachability probe is injected (no external target,
no sockets opened here). This is run-control logic, not a network client.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import Callable, List, Optional
class Policy(str, Enum):
HALT = "halt"
PAUSE_RESUME = "pause_resume"
class Outcome(str, Enum):
OK = "ok" # all required pools reachable at every check
INCONCLUSIVE = "inconclusive" # HALT fired (or PAUSE timed out → escalated)
RESUMED = "resumed" # PAUSE fired and the pool returned in time
@dataclass
class FrozenState:
"""The exact run state a PAUSE freezes so RESUME is bit-identical."""
churn_tick: int
seed_cursor: int
cell_id: str
@dataclass
class WatchdogReport:
policy: Policy
outcome: Outcome
checks: int = 0
outages: List[int] = field(default_factory=list) # ticks at which a pool was down
paused_ticks: int = 0
note: str = ""
@dataclass
class Watchdog:
"""Pre-registered connectivity failsafe.
``probe`` returns True iff every REQUIRED pool is reachable. It is injected so this
module opens nothing and reaches no target. ``pause_budget`` bounds how long a
PAUSE_RESUME will wait (in ticks) before escalating to HALT.
"""
policy: Policy
probe: Callable[[], bool]
pause_budget: int = 30
def check(self, state: FrozenState, report: WatchdogReport) -> Outcome:
"""Run one watchdog check at the current frozen state. Deterministic."""
report.checks += 1
if self.probe():
return Outcome.OK
report.outages.append(state.churn_tick)
if self.policy is Policy.HALT:
report.outcome = Outcome.INCONCLUSIVE
report.note = (
f"required pool unreachable at tick {state.churn_tick}; "
"HALT policy → cell inconclusive (pre-committed outcome)"
)
return Outcome.INCONCLUSIVE
# PAUSE_RESUME: clock + seed cursor are frozen in `state`; wait, bounded.
waited = 0
while waited < self.pause_budget:
waited += 1
report.paused_ticks += 1
if self.probe():
report.outcome = Outcome.RESUMED
report.note = (
f"paused {waited} tick(s) at churn_tick={state.churn_tick}, "
f"seed_cursor={state.seed_cursor}; resumed bit-identically"
)
return Outcome.RESUMED
report.outcome = Outcome.INCONCLUSIVE
report.note = (
f"PAUSE budget {self.pause_budget} exhausted at tick {state.churn_tick} "
"→ escalated to HALT (inconclusive)"
)
return Outcome.INCONCLUSIVE
def manifest_record(self) -> dict:
"""What to echo into manifest.json so the policy is provably pre-committed."""
return {"connectivity_failsafe": {"policy": self.policy.value,
"pause_budget_ticks": self.pause_budget}}