"""Connectivity failsafe for load-bearing node pools — DRAFT (next experiment only). Purpose (see prereg NEXT-EXPERIMENT-DRAFT §2): when node pools are *load-bearing* for the measured data (Track B/D), a run-control watchdog periodically probes every required pool and, on an outage, fires a **pre-registered** policy — chosen in advance and echoed into the manifest, never decided after seeing data. Policies: * HALT (default, integrity-first): abort the run, mark the cell ``inconclusive`` (a pre-committed outcome, so there is no optional-stopping bias), seal what exists. * PAUSE_RESUME (only when determinism is provable): freeze the churn clock + seed stream, wait (bounded) for the pool to return, then resume from the exact frozen state. Legitimate only because the seed formula makes the resumed sequence bit-identical. Why this did NOT apply to the sealed study: there, the phones were non-forwarding, so no measured data depended on connectivity. Here the pools ARE in the data path, so the watchdog protects real integrity. Self-contained + deterministic: the reachability probe is injected (no external target, no sockets opened here). This is run-control logic, not a network client. """ from __future__ import annotations from dataclasses import dataclass, field from enum import Enum from typing import Callable, List, Optional class Policy(str, Enum): HALT = "halt" PAUSE_RESUME = "pause_resume" class Outcome(str, Enum): OK = "ok" # all required pools reachable at every check INCONCLUSIVE = "inconclusive" # HALT fired (or PAUSE timed out → escalated) RESUMED = "resumed" # PAUSE fired and the pool returned in time @dataclass class FrozenState: """The exact run state a PAUSE freezes so RESUME is bit-identical.""" churn_tick: int seed_cursor: int cell_id: str @dataclass class WatchdogReport: policy: Policy outcome: Outcome checks: int = 0 outages: List[int] = field(default_factory=list) # ticks at which a pool was down paused_ticks: int = 0 note: str = "" @dataclass class Watchdog: """Pre-registered connectivity failsafe. ``probe`` returns True iff every REQUIRED pool is reachable. It is injected so this module opens nothing and reaches no target. ``pause_budget`` bounds how long a PAUSE_RESUME will wait (in ticks) before escalating to HALT. """ policy: Policy probe: Callable[[], bool] pause_budget: int = 30 def check(self, state: FrozenState, report: WatchdogReport) -> Outcome: """Run one watchdog check at the current frozen state. Deterministic.""" report.checks += 1 if self.probe(): return Outcome.OK report.outages.append(state.churn_tick) if self.policy is Policy.HALT: report.outcome = Outcome.INCONCLUSIVE report.note = ( f"required pool unreachable at tick {state.churn_tick}; " "HALT policy → cell inconclusive (pre-committed outcome)" ) return Outcome.INCONCLUSIVE # PAUSE_RESUME: clock + seed cursor are frozen in `state`; wait, bounded. waited = 0 while waited < self.pause_budget: waited += 1 report.paused_ticks += 1 if self.probe(): report.outcome = Outcome.RESUMED report.note = ( f"paused {waited} tick(s) at churn_tick={state.churn_tick}, " f"seed_cursor={state.seed_cursor}; resumed bit-identically" ) return Outcome.RESUMED report.outcome = Outcome.INCONCLUSIVE report.note = ( f"PAUSE budget {self.pause_budget} exhausted at tick {state.churn_tick} " "→ escalated to HALT (inconclusive)" ) return Outcome.INCONCLUSIVE def manifest_record(self) -> dict: """What to echo into manifest.json so the policy is provably pre-committed.""" return {"connectivity_failsafe": {"policy": self.policy.value, "pause_budget_ticks": self.pause_budget}}