diff --git a/.gitignore b/.gitignore index 9338690..4b72767 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ # build output /credrot /dist/ +/incredigo-map.html # never commit secrets or backup bundles (any sealer: age / hmac / openssl) *.credrot.enc @@ -11,6 +12,7 @@ # local-only design docs (kept on disk, never pushed upstream) docs/ROTATION.md +docs/ROTATION-REAL-CRED-RUNBOOK.md # editor / OS *.swp diff --git a/CLAUDE.md b/CLAUDE.md index bdee705..fc6264f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,19 +36,27 @@ Both must run **interactively or fully headless** (`$INCREDIGO_PASSPHRASE`, no T do the automatable rotations unattended, and emit a guided worklist (+ links) for the credentials that legitimately require a human. See `docs/ROTATION.md` Part II. -## Where development is (2026-06-15) +## Where development is (2026-07-14) -- **v1 shipped:** discover → migrate → expiry → sealed export/import. 8 scanners - (aws, env, netrc, ssh, docker, kube, git, file/`--path`); sealers age (default), - hmac, openssl. Tests green, `-race` clean. -- **Rotation: design + safety-spine + guided manual layer.** `internal/rotate` has - the `Rotator` interface and the **mandatory backup gate** (`Snapshot`); dry-run - `incredigo rotate` refuses `--execute`. The guided layer is built and **read-only**: - `internal/links` (offline change-password URL generation: curated table + RFC 8615 - well-known + host fallback), `internal/worklist` (secrets-free worklist), and - `internal/tui` (Bubble Tea walk-through) — surfaced as `incredigo worklist` and - `incredigo guide`. **Zero rotation drivers exist; nothing changes a credential at - any service yet** — keep it that way until explicitly authorized. +> Snapshot: [`docs/STATE-2026-07-14.md`](docs/STATE-2026-07-14.md) · plan: +> [`docs/ROADMAP.md`](docs/ROADMAP.md) · visual map: `incredigo-map.html`. + +- **v1 shipped:** discover → migrate → expiry → sealed export/import. 9 scanners + (aws, env, netrc, ssh, docker, kube, git, tea, file/`--path`); sealers age + (default), hmac, openssl. 179 tests green, `-race` clean. +- **Rotation: 22 drivers + wired Mode A execute spine.** All drivers self-register; + `rotate --execute` runs behind `INCREDIGO_ALLOW_EXECUTE=1` **and** the mandatory + verified-backup gate (`Snapshot`). Proof status is DATA (`internal/rotate/proofs.go` + mirrored in `docs/ROTATION-PROOFS.md`): **8 LIVE-VM** (postgres, mysql, redis, + mongo, wireguard, gitea, appsecret, k8s), **14 MOCK-ONLY**. **No real (host) + credential has been rotated yet** — every real rotation still requires explicit + per-credential authorization (safe-candidate ladder, ROADMAP M2). +- **Guided manual layer done:** `internal/links` + `internal/worklist` + + `internal/tui` → `incredigo worklist` / `incredigo guide` (read-only). +- **Phase B scaffolded, not wired:** 12 pwstore adapters, pwgen, staging; + `incredigo passwords pw-*` flow incomplete (ROADMAP M4). +- **Known weak spots:** sink 30% / cmd 4.3% coverage, no CI, lab-provision scripts + not in repo (ROADMAP M0–M1). ## Hard rules (do not violate — this tool holds real credentials) diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 0000000..96cd933 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,181 @@ +# incredigo — development roadmap to a stable, valuable rotation system + +Written 2026-07-14 against commit `d237098`. Current state: +[`STATE-2026-07-14.md`](STATE-2026-07-14.md). Ordering principle: **reliability of +the safety spine first, then real rotations one credential at a time, then the +headless turnover loop that is the actual product value, then Phase B.** + +## Definition of done ("stable, usable, adds security value") + +A user can run **one headless command** (cron/systemd-timer friendly) that: + +1. discovers credential sprawl (or reads gopass — Mode A), +2. produces a **verified sealed backup** before touching anything, +3. rotates every credential a proven driver claims (verify-new-before-revoke-old), +4. emits a secrets-free guided worklist for everything else, +5. leaves a redacted audit trail and a restorable backup chain, + +and every driver that can run `--execute` against a real service has a recorded +**LIVE-VM or LIVE-REAL proof** — no unproven code path can destroy a credential. + +--- + +## M0 — Housekeeping (small, do first) + +- [ ] Update `CLAUDE.md` "Where development is": 22 drivers exist, execute spine + wired behind `INCREDIGO_ALLOW_EXECUTE=1`, 8 LIVE-VM / 14 MOCK-ONLY, Phase B + scaffolded. The "zero drivers" claim actively misleads future sessions. +- [ ] Commit the pending `.gitignore` change; fix upstream tracking + (`master...origin/master [gone]`) and push to gitea when authorized. +- [ ] Gitignore `incredigo-map.html` (build artifact) or regenerate per release. +- [ ] Bring `lab-provision-*.sh` into the repo under `lab/` (they contain no + secrets — fake creds only). LIVE-VM proofs must be reproducible from a fresh + clone. Add `lab/README.md` mapping script → driver → proof. + +## M1 — Harden the safety spine (test debt on the critical path) + +The backup gate is only as trustworthy as the sealers. **This blocks all real +rotation work.** + +- [ ] `internal/sink` 30% → ≥85%: + - Round-trip per sealer (age/hmac/openssl): seal → open → byte-equal. + - Tamper tests: flip a byte in ciphertext/MAC → age & hmac MUST fail closed; + document openssl (unauthenticated) failure mode honestly. + - Wrong-passphrase → clean error, no partial plaintext. + - `Gopass` wrapper vs fake binary: insert/show/list/exists, stdin streaming + (assert no secret ever appears in argv or a temp file). + - Bundle framing: truncated stream, oversized path, zero-record bundle. +- [ ] `cmd/incredigo` 4.3% → meaningful integration coverage (fixture HOME + + `GOPASS_HOMEDIR` + fake gopass, pattern already exists in + `internal/rotate/backup_test.go`): + - scan/status/worklist happy paths; migrate --dedupe/--force. + - export→import round-trip through the real CLI. + - **Gate refusals**: `rotate --execute` without `INCREDIGO_ALLOW_EXECUTE=1`; + backup-verify failure aborts before any driver runs; missing passphrase. +- [ ] `internal/policy` 65.8% / `internal/tui` 57.1% → fill parsing and view gaps. +- [ ] CI on gitea (Actions): `go build ./... && go vet ./... && go test -race + ./...` + coverage floor per package (fail under 70%; sink/rotate/vault 85%). +- [ ] Add a `make` or `just` file: `build / test / race / cover / vm-proof + DRIVER=x / map`. + +Exit criteria: safety-critical packages ≥85%, CI green on push, gate-refusal +paths proven at the CLI level. + +## M2 — First real rotations (Mode A, safe-candidate ladder) + +Rule (unchanged): **one credential at a time, explicit authorization per +credential, verified backup + restore-drill first, verify-new-before-revoke-old.** + +- [ ] Add `ProofLiveReal` to `proofs.go` (+ docs mirror + PROOF column) so + promotions are recorded as data. +- [ ] Policy gate in `execute.go`: `--execute` only runs drivers with proof ≥ + LIVE-VM; MOCK-ONLY requires an explicit `--allow-mock-proven` opt-in flag. + Today nothing distinguishes them at execute time — closing this is the last + structural safety gap. +- [ ] Climb the ladder, promoting proofs as you go (lowest blast radius, fastest + recovery first): + 1. **appsecret** — local file, trivially reversible. + 2. **gitea** — self-hosted, token re-issuable in seconds, we own the server. + 3. **Self-hosted DB** (postgres/redis/mongo on own infra) — LIVE-VM drivers, + in-place ALTER, backup covers lockout. + 4. **Free-tier SaaS keys** (resend → vercel → sendgrid) — promotes MOCK-ONLY + drivers to LIVE-REAL with disposable keys. + 5. **cloudflare / flyio / twilio / gcp** — real accounts, still revocable from + the provider console if the driver misbehaves. + 6. **aws** — last: real IAM user with minimal policy, console as recovery path. +- [ ] Each rung: runbook entry (local ROTATION-REAL-CRED-RUNBOOK.md), audit log + retained, proofs.go + ROTATION-PROOFS.md updated in the same commit. +- [ ] Backup restore drill: before the first real rotation, prove + `import` of a sealed backup actually restores the store (not just that the + bundle verifies). + +Exit criteria: ≥5 drivers at LIVE-REAL, zero incidents, execute-gate enforces +proof levels. + +## M3 — Headless turnover loop (the product value) + +This is where incredigo stops being a toolbox and becomes "credential storage +turnover, automated." + +- [ ] `incredigo rotate --stale` — rotate only credentials the policy engine marks + Warn/Stale (wire `internal/policy` into plan selection). Expiry-driven + rotation is the whole point of the policy engine. +- [ ] Machine-readable output: `--json` for rotate/status, meaningful exit codes + (0 = clean, N = manual items remain, etc.) so cron/CI can react. +- [ ] Worklist emission on every headless run: automatable creds rotated, + remainder written to `rotate-worklist.md` (+ optional notification hook). +- [ ] **Backup chain turnover**: prune/rotate `~/.incredigo/rotation-backups/` + (keep N, verify oldest-retained still opens) — sealed backups are themselves + credential storage and must turn over. +- [ ] Ship `contrib/systemd/incredigo-rotate.{service,timer}` + cron example using + `INCREDIGO_PASSPHRASE` from a root-owned env file. +- [ ] End-to-end headless test in CI: fixture store → timer-style invocation → + assert rotated + worklist + audit + backup. + +Exit criteria: a weekly timer on a real machine rotates stale automatable creds +unattended and emails/writes the manual worklist. **This is the v2 "demo that +sells it."** + +## M4 — Phase B: browser/password-manager turnover + +Adapters and pwgen exist; wire the loop (design in BROWSER-ROTATION.md): + +- [ ] `pw-plan`: backup (manager export → sealed bundle) + generate new passwords + into staging (`~/.incredigo/pw-stage`, handles only). +- [ ] `pw-guide`: TUI walk-through per site — open change-URL, clipboard-inject the + staged password with auto-clear (decide + document the clipboard threat + model), human completes the site's form (never bypass MFA/CAPTCHA). +- [ ] `pw-commit`: only after human confirms the site accepted it → write back + in-place (CLI managers) or CSV re-import (tmpfs + shred, `--allow-csv`). +- [ ] Crash-safe staging: interrupted run resumes; stale stage entries expire. +- [ ] Prove end-to-end in the VM against a throwaway KeePassXC DB and a Firefox + CSV; record as LIVE-VM proof data (same pattern as drivers). + +## M5 — Long tail (only after M3 ships) + +- Browser-automation tier: go-rod driver for sites with stable change-password + forms (self-owned accounts only; secrets injected locally, never into a model). +- AI router as a **skill/MCP layer on top** — classify unknown creds, propose the + worklist ordering; never in the deterministic destructive hot path. +- New scanners toward the VISION.md 20-credential list: gcloud ADC, .npmrc/.pypirc, + LLM provider keys. +- Distribution: versioned releases on gitea, `go install`, signed binaries. + +--- + +## Ideal end-state asset inventory + +**In-repo (code + data):** +- All of M1–M4; `lab/` provisioning scripts; `contrib/systemd/`; Makefile/justfile; + CI workflow; proofs.go with a LIVE-REAL column that grows monotonically. + +**Docs (ICM layering, already close):** +- L1 router: `CLAUDE.md` (kept current — M0). +- L2 mission/architecture: `VISION.md`, `DESIGN.md`, `docs/MARKET.md`. +- L3 canonical reference: `docs/ROTATION-PROOFS.md` (mirror of data), + `docs/SECURITY.md`, `docs/ADDING_A_SCANNER.md`, this `ROADMAP.md` + (+ local-only ROTATION.md / RUNBOOK). +- L4 journal: dated `STATE-*.md` snapshots (this pattern), FINDINGS docs, + `incredigo-map.html` regenerated per milestone. + +**Claude Code layer (keep the LLM out of the hot path):** +- A project skill (e.g. `incredigo-ops`) encoding the M2 ladder as a runbook: + pick next candidate → provision VM proof → run gated rotation → promote + proofs.go → update docs. Deterministic steps in Go/bash; the skill only + orchestrates and asks for authorization. +- Optionally a `/loop`-able health check: run `incredigo status --json` and surface + stale creds in the morning briefing. + +**Infra:** +- Gitea CI runner executing the -race suite + coverage gates. +- Reproducible Multipass VM harness (`make vm-proof DRIVER=x`). +- A real machine running the M3 systemd timer — dogfooding is the proof. + +## Suggested execution order (next 5 work sessions) + +1. M0 entirely + sink sealer tests (biggest safety ROI). +2. CLI integration tests + CI pipeline. +3. `ProofLiveReal` + execute-gate proof policy + restore drill. +4. Ladder rungs 1–3 (appsecret, gitea, one self-hosted DB) — with per-cred + authorization. +5. `rotate --stale` + JSON output + systemd timer → first headless turnover demo. diff --git a/docs/STATE-2026-07-14.md b/docs/STATE-2026-07-14.md new file mode 100644 index 0000000..2316d59 --- /dev/null +++ b/docs/STATE-2026-07-14.md @@ -0,0 +1,115 @@ +# incredigo — state snapshot (2026-07-14) + +Point-in-time assessment at commit `d237098`. Companion doc: [`ROADMAP.md`](ROADMAP.md) +(how to proceed). Visual map: `incredigo-map.html` (repo root, regenerate with +`/project-map`, build artifact). + +## TL;DR + +- **v1 custody pipeline is done and solid**: discover → vault → migrate → status → + sealed export/import. 179 tests green across 13 packages, `-race` clean. +- **Rotation is much further along than CLAUDE.md claims**: 22 drivers exist and + self-register; the Mode A `--execute` spine is wired behind + `INCREDIGO_ALLOW_EXECUTE=1` + the mandatory verified-backup gate. 8 drivers are + LIVE-VM proven, 14 MOCK-ONLY. **No real (host) credential has been rotated yet** — + that remains explicitly gated pending per-credential authorization. +- **Guided manual layer works end-to-end**: `worklist` + `guide` (TUI) with offline + change-URL derivation (49 curated sites + RFC 8615). +- **Phase B (browser/password-manager turnover) is scaffolded, not wired**: + 12 pwstore adapters + pwgen + staging exist; `passwords pw-*` subcommands are in + design phase. +- **Weakest links**: `internal/sink` 30% coverage (the sealers — safety-critical), + `cmd/incredigo` 4.3% (no CLI integration tests), lab-provision scripts referenced + by `proofs.go` are not in the repo, no CI. + +## Component inventory + +| Layer | Package | What exists | Status | +|---|---|---|---| +| Custody | `internal/vault` | memguard arena (mlock/DONTDUMP/zeroize), Handle indirection | done | +| Discover | `internal/discover` | 9 scanners: aws, env, netrc, ssh, docker, kube, git, tea, file(`--path`) + service-prefix detection (GitHub PAT, Stripe) | done | +| Custody/backup | `internal/sink` | gopass stdin streaming, bundle framing, sealers age/hmac/openssl | done, under-tested | +| Policy | `internal/policy` | expiry engine, YAML, per-kind overrides | done | +| Audit | `internal/audit` | append-only redacted JSONL | done | +| Rotation | `internal/rotate` | Rotator iface, registry, **mandatory backup gate**, Mode A execute spine, dry-run noop, 22 drivers, proofs-as-data | wired, no LIVE-REAL proof yet | +| Blast radius | `internal/blast` | read-only consumer map from non-secret markers | done | +| Guided layer | `internal/links` / `worklist` / `tui` | offline change-URLs, secrets-free checklist, Bubble Tea walk-through | done | +| Phase B | `internal/pwstore` / `pwgen` / `cmd/.../passwords.go` | 12 manager adapters (bw, op, keepassxc, pass, keychain, chrome/firefox + 8 CSV managers), staging, generation | scaffolded, not wired end-to-end | +| CLI | `cmd/incredigo` | scan, migrate, status, export, import, rotate, worklist, guide, passwords | done except passwords | + +## Rotation driver proof matrix (from `internal/rotate/proofs.go` — single source of truth) + +**LIVE-VM (8)** — real target software in sandbox VM: postgres, mysql (MariaDB), +redis, mongo (mongod 8.0), wireguard, gitea, appsecret, k8s (k3s v1.35). + +**MOCK-ONLY (14)** — httptest/emulator/fake-binary only: aws (moto), sshkey, +openwrt, mullvad, cloudflare, ghactions, gitlab, npm (real registry needs password +— known caveat), gcp (real RS256 JWT self-auth), twilio, flyio, resend, vercel, +sendgrid. + +**LIVE-REAL (0)** — the level doesn't exist yet in code; adding it is roadmap M2. + +All drivers follow create→verify→revoke (or in-place ALTER for DBs) and contain +only real service code (no simulation branches — proof status is data, per project +rule). + +## Test coverage (go test ./... -cover, 2026-07-14) + +| Package | Coverage | Assessment | +|---|---|---| +| internal/vault | 100.0% | ✅ | +| internal/worklist | 100.0% | ✅ | +| internal/links | 93.8% | ✅ | +| internal/blast | 92.1% | ✅ | +| internal/audit | 90.5% | ✅ | +| internal/pwgen | 84.1% | ✅ | +| internal/pwstore | 78.4% | ok | +| internal/rotate | 76.4% | ok (26s runtime: fake bins + httptest) | +| internal/discover | 71.9% | ok | +| internal/policy | 65.8% | ⚠ parsing edge cases | +| internal/tui | 57.1% | ⚠ Update() only, View()/Run() untested | +| **internal/sink** | **30.0%** | ❌ sealers + gopass wrapper barely tested — safety-critical gap | +| **cmd/incredigo** | **4.3%** | ❌ no CLI integration tests | + +Test quality where coverage exists is high: table-driven, secret-leak assertions, +real-cutover proofs (old cred stops working), fake gopass/psql binaries, +in-process SSH servers, httptest emulators. + +## Demos buildable today (zero new code) + +1. **Sprawl audit (safest, most impressive)** — on any machine: + `incredigo scan` → `incredigo status` → `incredigo worklist --out rotate.md` → + `incredigo guide`. Read-only, no secrets printed, TUI opens change-URLs. +2. **Custody round-trip** — isolated store (`GOPASS_HOMEDIR=$(mktemp -d)` + + throwaway GNUPGHOME): fixture `.env`/`.aws` → `migrate` → `export` (age) → + wipe → `import` → verify. Demonstrates "no plaintext ever on disk". +3. **Rotation dry-run + blast radius** — `rotate --dry-run --blast` over a fixture + repo: shows the backup gate firing, the plan table with PROOF column, and which + files consume each credential. +4. **LIVE-VM rotation** — Multipass sandbox VM (see local runbook): provision + postgres/redis/gitea with fake creds, `rotate --execute` with + `INCREDIGO_ALLOW_EXECUTE=1`, show old cred dead / new cred live / sealed + backup restorable. The full end-to-end story, zero real-cred risk. +5. **Phase B ingest** — `passwords pw-scan` against a fixture browser CSV / + throwaway KeePassXC DB (scan only; plan/commit not wired). + +## Known gaps & risks + +1. **`internal/sink` 30% coverage** — the sealers guard every backup the rotation + gate depends on. Tamper/round-trip/wrong-passphrase tests missing. Highest-value + test work in the repo. +2. **No CLI integration tests** (cmd 4.3%) — flag plumbing, gate refusals + (`--execute` without env var, no-passphrase, backup-verify failure) are untested + at the command level. +3. **Lab-provision scripts not in repo** — `proofs.go` cites + `lab-provision-{pg,dbclones,wg,gitea,appsec,mongo,k8s}.sh` but they live only in + the VM/runbook. LIVE-VM proofs are not reproducible from a fresh clone. +4. **No CI** — gitea remote exists; branch upstream is gone (`master...origin/master + [gone]`); nothing runs build/vet/test -race on push. +5. **CLAUDE.md "Where development is" is stale (2026-06-15)** — claims "zero + rotation drivers exist"; reality is 22 drivers + wired execute spine. +6. **Phase B not wired** — adapters exist but pw-plan→pw-commit flow incomplete. +7. **No scheduled turnover** — `status` knows what's stale; nothing acts on it + headlessly yet (roadmap M3). +8. **npm caveat** — real registry token creation requires the account password; + driver can never leave MOCK-ONLY without a design decision.