Files
incredigo/docs/FINDINGS-2026-06-16.md
T
leetcrypt 9109da7ae4 rotate: promote k8s + mongo to LIVE-VM via real-target POCs
Ran live cutover POCs in the sandbox VM against the real target software and
promoted both drivers from MOCK-ONLY to LIVE-VM in the proof table (data, not
driver behaviour):

- k8s: proven against real k3s v1.35 kube-apiserver + token controller
  (lab-provision-k8s.sh). The real apiserver serves a self-signed cert, so the
  driver gained CA-pinned TLS — blob params ca= (base64 PEM, pinned as the only
  trusted root) and insecure=1 (lab-only escape hatch), routed through a new
  clientFor(cr). Neither is test-awareness; a real deployment configures the same
  CA. Added TestK8sTLSTrust covering CA-pinned / insecure / untrusted-rejection
  and the blob round-trip.
- mongo: proven against real mongod 8.0 (lab-provision-mongo.sh).

npm stays MOCK-ONLY by decision (registry.npmjs.org not self-hostable; real
create-token requires the account password in the body) — documented as a
contract gap rather than faking a LIVE-VM. gcp/twilio/flyio remain MOCK-ONLY.

Live POCs surfaced real-target-only behaviour now recorded in the docs: the
apiserver's ~10s successful-auth cache (old token kept working ~7s after Secret
deletion) and real mongosh's stdout prompt. Full suite 104 green, -race clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-18 21:50:21 -07:00

393 lines
25 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Findings — controlled-environment test run (2026-06-16)
Source: `~/incredigo-lab-stage/lab-harness.sh` (fresh isolated gopass store + fake,
canary-tagged inputs for all 8 scanners). FAKE creds only; never touches a real store.
## Live rotation POC — first real driver (postgres), 2026-06-16
**Result: real end-to-end cutover proven** in the `incredigo-sbx` Multipass VM (real
gopasspw/gopass 1.15.16, real PostgreSQL 16, isolated `GOPASS_HOMEDIR`/`GNUPGHOME`,
FAKE `labapp@labdb` role).
Flow exercised by the shipped binary via `incredigo rotate --execute`
(gated: `INCREDIGO_ALLOW_EXECUTE=1` + `--execute`):
1. **Backup gate** sealed + verified 1 entry → `~/.incredigo/rotation-backups/<ts>.age` before any change (Hard Rule #1).
2. **Rotate** ran `ALTER USER labapp WITH PASSWORD '<48-hex>'` over the OLD connection.
3. **Verify(new)**`SELECT 1` with the new secret passed.
4. **Store** the rebuilt DSN back into gopass at the same entry; **re-read + Verify(stored)** passed.
5. **RevokeOld** = no-op (in-place ALTER is itself the cutover).
Cutover assertions (all passed):
- OLD password (`oldpw835`) **rejected** after rotation — `FATAL: password authentication failed`.
- NEW 24-byte hex password (read back from gopass) **authenticates**.
- Audit log (`rotate-execute`) contains **no password substring** (old or new).
Spine: `internal/rotate/execute.go` (`Execute`), driver `internal/rotate/postgres.go`
(self-registers via `init`), sourcing `cmd/incredigo` `gopassCredsForExecute`
(Mode A — reconstructs `Source` from the gopass path segment), persistence
`sink.Gopass.Show`/`InsertAt`. Unit proof of the same cutover (no live DB):
`internal/rotate/postgres_test.go:TestPostgresRotateRealCutover`. Full suite: 45 green.
## Live rotation POC — gitea PAT driver (provider-API pattern), 2026-06-17
**Result: real create-new → verify → revoke-old cutover proven** against a real
Gitea 1.25.0 (sqlite, http on loopback) inside the `incredigo-sbx` VM, isolated
gopass store, self-owned `labadmin` account. This is the **first provider-API
driver** and the first whose `RevokeOld` is a real destructive API call.
Provision: `~/incredigo-lab-stage/lab-provision-gitea.sh` (stands up Gitea, mints a
seed token via basic auth, seeds the driver-ready blob into gopass). Flow exercised
by the shipped binary via `incredigo rotate --execute` (gated):
1. **Backup gate** sealed + verified 1 entry before any change (Hard Rule #1).
2. **Rotate** `POST /api/v1/users/labadmin/tokens` (basic auth) → new token.
3. **Verify(new)** `GET /api/v1/user` with the new token → 200, `login=labadmin`.
4. **Store** the rebuilt blob back into gopass; **re-read + Verify(stored)** passed.
5. **RevokeOld** `DELETE …/tokens/<old-name>` — a real token deletion.
Cutover assertions (all passed):
- OLD token → `GET /user` returns **401** after rotation (revoked).
- NEW token (read back from gopass) → **200**, `login=labadmin`.
- Gitea token list now shows only `incredigo-rotated-<ts>`; the seed token is gone.
- Audit log (`rotate-execute`) contains **no token substring** (old or new); old ≠ new.
Spine: `internal/rotate/gitea.go` (single-line `https://user:token@host/?name=&pw=`
blob so it round-trips the existing single-line gopass sink; `authManage` prefers
basic auth when a mgmt password is present, else token auth). Discovery for the
gitea PAT type: `internal/discover/tea.go` (Gitea `tea` CLI config → driver-ready
blob, Source `gitea`). Unit proof of the same enforced cutover (httptest Gitea
emulator that 401s a revoked token): `internal/rotate/gitea_test.go`.
## SSH key driver (unit-proven cutover), 2026-06-17
`internal/rotate/sshkey.go` mints a new ed25519 keypair **entirely in memory**
(golang.org/x/crypto/ssh; private key never touches disk — Hard Rule #3), installs
only the new PUBLIC key into `authorized_keys`, verifies by SSH-dialing the
authority, and `RevokeOld` removes the old public key (atomic rewrite). Proven by an
**in-process SSH server** test: old key rejected after revoke, new key works, no
`PRIVATE KEY` material in `authorized_keys` (`internal/rotate/sshkey_test.go`).
Discovery broadened (`internal/discover/ssh.go`): content-based key detection (not
just `id_*`) + `~/.ssh/config` `IdentityFile` parsing. Live VM POC not yet run.
Full suite after all three drivers: **50 green**, `-race` clean on
rotate/sink/vault/discover.
## Live rotation POC — AWS IAM access key driver (cloud-key pattern), 2026-06-17
**Result: real create-new → verify → revoke-old cutover proven** against a
self-hosted **moto 5.2.2** mock AWS (`moto_server` on http://127.0.0.1:5000) inside
the `incredigo-sbx` VM, isolated gopass store, self-created IAM user `labapp`. This
is the cleanest provider-API driver: the OLD credential **self-identifies** — an
access key's `AccessKeyId` is non-secret and rides in the secret blob, so RevokeOld
deletes exactly the right key (no name-capture dance Gitea needed).
Implementation note: the driver speaks the AWS **Query protocol** (form-POST
`Action=…&Version=…`) signed with **hand-rolled SigV4** (`crypto/hmac`) — zero new
go.mod deps, keeping the static binary lean (the AWS SDK v2 would have added ~50
indirect deps). Spine: `internal/rotate/aws.go`. Secret blob is the single line
`aws://<AccessKeyId>:<SecretAccessKey>@aws/?region=<r>[&endpoint=<url>]` so it
round-trips the single-line gopass sink; `endpoint=` overrides the API base for
moto/LocalStack, absent → real `iam/sts.amazonaws.com`. Discovery broadened:
`internal/discover/aws.go` now emits that driver-ready blob (was the bare secret) and
resolves region from env / `~/.aws/config`.
Flow exercised by the shipped binary via `incredigo rotate --execute` (gated):
1. **Backup gate** sealed + verified 1 entry before any change (Hard Rule #1).
2. **Rotate** IAM `CreateAccessKey` (no UserName → moto resolves the calling
identity from the signing key) → new key pair.
3. **Verify(new)** STS `GetCallerIdentity` with the new key → ARN returned.
4. **Store** the rebuilt blob into gopass; **re-read + Verify(stored)** passed.
5. **RevokeOld** IAM `DeleteAccessKey` targeting the OLD `AccessKeyId` — a real delete.
Cutover assertions (all passed, `lab-verify-aws.sh`):
- After rotate, IAM `ListAccessKeys(labapp)` shows **only** the new key; the old
`AccessKeyId` is **gone** at the provider. old ≠ new.
- New key (read back from gopass) authenticates; audit log (`rotate-execute`) carries
**no AccessKeyId and no SecretAccessKey**.
**moto enforcement caveat (investigated, important).** moto's STS
`GetCallerIdentity` does **not** validate access-key existence — a *deleted* key
still returns a (generic `user/moto`) ARN rather than a 403. So against moto, the
driver's STS-based Verify cannot itself prove "old key dead"; the live cutover proof
is therefore **IAM `ListAccessKeys`** (ground truth that the delete took effect),
asserted in `lab-verify-aws.sh`. The *enforced* 403-on-revoked-key path (what real
AWS does) is proven instead by the **unit test**: `internal/rotate/aws_test.go`'s
`awsEmu` reads the `AccessKeyId` from the SigV4 `Credential=` header and rejects any
key not currently present, so `TestAWSRotateRealCutover` asserts the old key returns
403 after revoke, the new key still authenticates, and old ≠ new. The driver's Verify
(plain "ARN non-empty") is correct for real AWS, which **does** 403 an invalid key;
no moto-specific leniency was baked into the driver.
Provision: `~/incredigo-lab-stage/lab-provision-aws.sh` (starts moto, creates
`labapp` + seed key via boto3, seeds the driver-ready blob into isolated gopass);
verify: `~/incredigo-lab-stage/lab-verify-aws.sh`. Unit proofs:
`TestAWSRotateRealCutover` (enforced cutover), `TestAWSSecretRoundTrip` (blob carries
`/`+`=` secrets intact), `TestSigV4HeadersStable` (Authorization shape).
Full suite after four drivers: **53 green**, `-race` clean on
rotate/sink/vault/discover.
## VPN / router / DB-clone batch — 5 drivers, 3 live cutovers, 2026-06-18
Approved expansion (VPN + router secondary persona, plus Tier-1 DB clones). Five new
one-file drivers, each with a table-driven unit test (cutover proof + leak-check). Three
were additionally proven **live** in the `incredigo-sbx` VM against real services; one is
unit-proven with a live QEMU POC deferred; one is unit-only by design.
| Driver | Pattern | Live POC | Proof |
|--------|---------|----------|-------|
| `mysql` | in-place DB password | **PASS** (real MariaDB) | `lab-provision-dbclones.sh` |
| `redis` | in-place `CONFIG SET requirepass` | **PASS** (real redis-server) | `lab-provision-dbclones.sh` |
| `wireguard` | local keypair regen + peer propagate | **PASS** (real `wg`) | `lab-provision-wg.sh` |
| `openwrt` | in-place SSH `passwd` | deferred (QEMU on 1-CPU VM) | unit: in-process SSH server |
| `mullvad` | provider-API device key | n/a (needs paid acct) | unit: httptest TLS emulator |
**MySQL + Redis live cutover (`lab-provision-dbclones.sh`, all assertions PASS).** Real
`mariadb-server` + `redis-server`, isolated gopass, fake `labapp@labdb` and a fake redis
`requirepass`. Backup gate sealed+verified first; `rotate --execute` flipped both; the
OLD password is **rejected** and the NEW one (read back via `gopass show -o`)
**authenticates** for each engine.
> **Driver gap found & fixed — MariaDB unprivileged self-rotation.** MariaDB rejects
> `ALTER USER CURRENT_USER() IDENTIFIED BY …` for a non-admin (ERROR 1227 — it demands the
> global `CREATE USER` privilege even for one's *own* account), which is exactly the
> target persona's situation (a vibe coder's DB user is scoped to one schema with
> `GRANT ALL ON app.*`). The MySQL-only `USER()` spelling also fails on MariaDB (ERROR
> 1064). Fix (`internal/rotate/mysql.go`): try a fallback chain — `ALTER USER
> CURRENT_USER()` (MySQL self-service) → `SET PASSWORD = PASSWORD('…')` (MariaDB / MySQL
> 5.7) → `SET PASSWORD = '…'` (MySQL 8 / MariaDB ≥10.4); first to succeed wins. After the
> fix the live POC passed on MariaDB.
**WireGuard live cutover (`lab-provision-wg.sh`, all assertions PASS).** Real
`wireguard-tools`: `wg0` = ours, `wg1` = the peer that trusts us. Seeds `wg0`'s old
private key into isolated gopass, `rotate --execute` mints a new clamped curve25519
keypair, applies the new private key to `wg0` (via stdin → `wg set wg0 private-key
/dev/stdin`, never argv), and re-points the peer to the new public key (carrying over
allowed-ips). Assertions: private key changed; peer trusts NEW pubkey; peer **dropped**
OLD pubkey; and the headline compatibility proof — **`wg0`'s live interface public key
equals the public key our Go code derived**, i.e. our in-process curve25519 derivation
matches real `wg pubkey`.
> **POC scripting gotcha (Mode A prefix).** Mode A reconstructs the driver `Source` from
> the **first** path segment after `--prefix` (`imported/<source>/<slug>`). The WG POC
> first ran with `--prefix imported/wireguard/`, which made `Source="wg0"` (no driver
> Detected it → ROTATED 0). Fix: point `--prefix` at the `imported/` **root** so
> `imported/wireguard/wg0` resolves `Source="wireguard"`. This is a harness fix, not a
> driver bug, but worth recording: narrowing the prefix below the source dir breaks
> source recovery.
**OpenWrt (unit-proven, live deferred).** `internal/rotate/openwrt.go` dials the router
with the OLD password and runs BusyBox `passwd`, feeding the new password twice on the
SSH-encrypted stdin (never argv); setting the password IS the cutover (RevokeOld no-op).
Proven by an **in-process SSH server** (`openwrt_test.go`) with a mutable current
password: old password stops authenticating after rotate, new works, no secret in argv or
error strings. A live OpenWrt-in-QEMU POC is the heaviest (full-system emulation on the
1-CPU sandbox VM) and is **deferred**.
**Mullvad (unit-only by design).** `internal/rotate/mullvad.go` is the provider-API
device-key pattern: fetch an access token from the account number, `POST
/accounts/v1/devices {pubkey}` to register a freshly-derived WireGuard key, verify the
device lists that exact pubkey, then `DELETE` the old device. Proven against an httptest
**TLS** emulator (`mullvad_test.go`) that inspects every request body for the private key
(never sent) and 404s a deleted device. No live POC — that needs a real paid Mullvad
account; the API contract is faithfully emulated instead.
**Discovery broadened (Hard-rule standing directive — dynamic discovery per type).** The
env scanner (`internal/discover/env.go`) now tags DB connection URLs by scheme:
`postgres`/`postgresql``postgres`, `mysql`/`mariadb``mysql`, `redis`/`rediss`
`redis` (requires a `user:password` authority), so `DATABASE_URL`/`REDIS_URL`/`MYSQL_URL`
in a `.env` route straight to the matching in-place driver, kept regardless of the
secret-name heuristic. Regression: `scanners_test.go:TestEnvScanner`.
Full suite after the batch: **68 green**, `-race` clean on rotate/sink/vault/discover.
## SaaS-token / app-signing batch — 4 drivers + proof manifest, 2026-06-18
Four more drivers, each one-file + table-driven test (cutover proof + leak-check). User
direction this batch: **keep real-rotation code unmistakably distinct from mock code, and
record the real-vs-mock distinction as DATA, not behaviour.** Implemented as a single Go
table (`internal/rotate/proofs.go`) + a human mirror (`docs/ROTATION-PROOFS.md`), surfaced
as a **PROOF** column in `incredigo rotate` so a mock-only driver can never be mistaken for
a live one.
- **gitlab** (`gitlab.go`, MOCK-ONLY) — GitLab PAT via `POST
/api/v4/personal_access_tokens/self/rotate` (`PRIVATE-TOKEN` header). GitLab mints the
new token *and revokes the caller's old one in the same call* → in-place, `RevokeOld`
no-op. Blob `gitlab://<host>/?tok=<token>`. The `httptest` emulator enforces the revoke,
so the unit test's `Verify(old)`-fails assertion proves a real cutover. GitLab CE is too
heavy for the VM → MOCK-ONLY.
- **cloudflare** (`cloudflare.go`, MOCK-ONLY) — Cloudflare API token *value roll* via
`PUT /client/v4/user/tokens/{id}/value` (Bearer). Same token id, fresh value, previous
value dead → in-place, `RevokeOld` no-op; keeps id/policies/name stable. Blob
`cloudflare://<host>/?id=<id>&token=<val>`. Verify via `/tokens/verify` (`status:active`).
- **ghactions** (`ghactions.go`, MOCK-ONLY) — GitHub Actions repo secret. Fetches the repo
libsodium public key (`actions/secrets/public-key`), seals a fresh value with
`crypto_box_seal` (`golang.org/x/crypto/nacl/box.SealAnonymous`, no new go.mod dep), and
`PUT`s it over the existing secret name (overwrite = cutover) → `RevokeOld` no-op. The
rotated material is the **secret value** (`val=`), NOT the management `pat=` that
authenticates the call. GitHub never returns secret values, so Verify is an
existence/authorization check (200 on the secret), not a value match. The emulator owns
the box keypair and **decrypts the sealed PUT**, asserting the decrypted value equals the
driver's new blob value — proving the encryption is real, not simulated.
- **appsecret** (`appsecret.go`, **LIVE-VM**) — LOCAL app signing secret (Django
`SECRET_KEY`, Rails `secret_key_base`, JWT/session secrets, …). Generates a fresh
64-hex value and rewrites it **in place across every target file** (atomic temp+rename,
mode preserved), keyed on the literal old value so it is file-format-agnostic
(.env/yaml/json/toml). In-place ⇒ `RevokeOld` no-op; Verify re-reads every file asserting
new-present / old-absent. Blob `appsecret://local/?key=&val=&path=…[&path=…]`. A
`minSecretLen` (12) guard refuses to key a replacement on a short/common string
(mass-corruption guard), and a zero-replacement result is an error (never claim a
rotation that didn't happen). Errors scrub the value via `redactErr`.
**appsecret discovery (`env.go`).** An exact-match set of app-signing key names
(`SECRET_KEY`, `SECRET_KEY_BASE`, `JWT_SECRET`, `APP_KEY`, …) now tags those `.env` entries
`Source="appsecret"` with a self-contained blob carrying the value AND the file path, so the
local rotator is driver-ready straight from a scan. Generic API keys (STRIPE_API_KEY) stay
plain env secrets. Regression: `scanners_test.go:TestEnvScannerAppSecret`.
**appsecret live VM cutover (LIVE-VM).** `lab-provision-appsec.sh` wrote two real config
files (`labapp/.env` 0600, `labworker/config.yaml` 0644) sharing one `SECRET_KEY`, seeded
the blob into isolated gopass, and ran the shipped binary `incredigo rotate --execute
--prefix imported/`. All assertions PASSED: backup gate sealed+verified first; both files
rewritten with a fresh 64-hex value; OLD value gone from both; unrelated `PORT=8080`
preserved; 0644 mode preserved. The run's table showed `appsecret LIVE-VM` — confirming the
new PROOF column renders in the real binary.
**Honesty taxonomy (the point of the manifest).** Running in the VM does NOT make a driver
LIVE-VM — only the *real target software* validating the cutover does. So this batch is
3× MOCK-ONLY (gitlab/cloudflare/ghactions: no self-host / heavy CE) + 1× LIVE-VM
(appsecret: real local files). AWS remains the canonical example: it ran in the VM but only
against `moto` (a mock), so it is MOCK-ONLY. Promoting a MOCK-ONLY driver needs only a real
service + re-run — zero driver code changes (the drivers contain no test awareness).
Full suite after the batch: **82 green**, `-race` clean on rotate/sink/vault.
## Service-API batch 2 — 6 drivers (mongo, npm, gcp, twilio, flyio, k8s), 2026-06-18
Six more rotation drivers, each **real-service code only** (no test awareness; tests inject
just an `HTTPClient`/`Bin`), validated against protocol-enforcing emulators:
- **mongo** (in-place DB) — `changeUserPassword` over the OLD authenticated connection is
itself the cutover; `RevokeOld` is a no-op. NO secret on argv: the URI handed to `mongosh`
carries host/db only, both passwords go in on stdin, captured output is scrubbed. Fake
`mongosh` stub authenticates against a state-file password → proves old password dies.
- **npm** (provider-API create→verify→revoke) — `POST /-/npm/v1/tokens` (Bearer old) →
`/-/whoami` (Bearer new) → `DELETE …/tokens/token/<key>`. Blob records the token `key=`
for deletion; degrades to guided when absent.
- **gcp** (provider-API, service-account KEY) — mints an **RS256 JWT signed with the SA's own
private key** (`crypto/rsa`, stdlib, no new dep), exchanges it for an access token, then
`CreateServiceAccountKey` → GET serviceAccount (verify) → `DeleteServiceAccountKey`. The
emulator **verifies the JWT signature** against the key's public key, so the test proves
genuine signing, not a stub. SA JSON round-trips base64-wrapped in the one-line blob.
- **twilio** (provider-API) — `POST …/Keys.json` (Basic old) → GET key (Basic new) →
`DELETE …/Keys/<sid>.json`. Basic-auth username=KeySid, password=secret.
- **flyio** (provider-API, GraphQL) — a **managing token** mints/deletes the **deploy token**
(the rotated value) via `createLimitedAccessToken` / `deleteLimitedAccessToken`; verify
runs the app query as the new deploy token. Same "auth rotates a value" shape as ghactions.
- **k8s** (provider-API) — creates a new `kubernetes.io/service-account-token` Secret, waits
for the controller to populate `.data.token`, GETs the SA to verify, then DELETEs the old
Secret (which invalidates the old token).
Each emulator enforces credential validity and the tests assert `Verify(old)` fails
after revoke, plus a leak canary (no token/secret/`PRIVATE KEY` in Identity/Source/Location).
### Live-VM POCs run (2026-06-18/19) → mongo + k8s promoted to LIVE-VM
- **mongo → LIVE-VM** (`lab-provision-mongo.sh`): real `mongod` 8.0 from the official apt
repo; `labapp` (admin, `userAdminAnyDatabase`) rotates its own password; old password
rejected, new works. *Surfaced by the real target:* real `mongosh` echoes a `labdb> ` prompt
to stdout on a pipe and `db.auth` returns an object not a bool in 2.8 — the fake-mongosh stub
did neither, so the assertion had to filter output to the `AUTH-OK`/`AUTH-FAIL` token.
- **k8s → LIVE-VM** (`lab-provision-k8s.sh`): real **k3s v1.35** kube-apiserver + token
controller; `labsa` (RBAC: create/delete/get Secrets + get its SA) rotates its legacy
token Secret; old token → `401`, new → `200`. *Surfaced by the real target:* (a) the
apiserver serves a self-signed cert, so the driver needed CA-pinned TLS — added `ca=`
(base64 PEM) + an `insecure=1` lab-only escape hatch to the blob; (b) the apiserver caches
*successful* auths ~10s (`cachedTokenAuthenticator`), so the deleted token kept working until
the cache expired (~7s) — the POC polls past it. Both are real-deployment config, not
test-awareness.
- **npm stays MOCK-ONLY** (decided, not pending): `registry.npmjs.org` isn't self-hostable and
real `POST /-/npm/v1/tokens` requires the **account password in the body** (the driver sends
only Bearer old-token). Documented as a contract gap in `docs/ROTATION-PROOFS.md` rather than
faking a LIVE-VM. gcp/twilio/flyio remain MOCK-ONLY (no self-host).
Full suite after batch 2 + POCs: **104 green**, `-race` clean on rotate/sink/vault.
## Summary
- Harness result: **31/31 PASS**, including the global secret-leak canary (no CANARY
token or `PRIVATE KEY` material reached any command output).
- One **real defect** found by manual triage of the backup gate, since fixed.
## Defect 1 — backup gate silently under-covered dot-segment entries (FIXED)
**Severity:** high — a silent violation of Hard Rule #1 (backup before rotate).
**Symptom.** After `migrate`, the store held 10 imported credentials, but the rotate
dry-run backup gate sealed/verified only **7**. The 3 missing were the `.env`-derived
entries: `imported/env/.env/{api_token,database_url,sendgrid_api_key}`.
**Root cause.** `internal/sink/gopass.go:slug()` lower-cased and space-collapsed the
credential Identity but **kept leading dots**. An env credential has
`Identity = ".env / API_TOKEN"`, so `StorePath` became
`imported/env/.env/api_token`. The `.env` path segment is dot-prefixed, and gopass
**hides dot-prefixed directories/leaves from `ls --flat`**. `bundle.ListPaths` (and the
backup gate that consumes it) enumerate via `ls --flat`, so those 3 entries were
invisible — never sealed, never verified.
**Why it was silent.** The gate's integrity check compares *sealed == verified* counts.
Both were 7, so 7==7 passed. The gate had no notion of *expected* count, so
under-coverage produced no error.
**Fix.** `slug()` now strips leading dots per `/`-separated path segment, so
`.env` → `env` (`imported/env/env/api_token`). Entries are now enumerable and the gate
covers all of them. Regression test: `internal/sink/bundle_test.go:TestStorePathNoDotSegment`
asserts no produced StorePath contains a dot-prefixed segment. Full suite: 42 tests
green, `-race` clean on `sink`/`rotate`.
## Root cause of the *lapse* (why it stayed silent) and prevention
The slug bug was the trigger, but it stayed silent because of a deeper flaw in the gate:
`rotate.Snapshot` asserts `verified == res.n`, and **both counts descend from the same
`gopass ls --flat` enumeration** (`ExportTo` → `ListPaths`, bundle.go). A blind spot in
that enumeration is inherited by both sides, so the comparison can never catch it. The
check proves *round-trip integrity* (the bundle re-reads to the same count it sealed), not
*coverage* (the bundle holds every credential in the store). **A safety gate that compares
two values derived from the same source cannot detect an error in that source.**
Prevention, layered:
1. **Source fix (done).** `slug()` strips leading dots → incredigo can no longer write any
path that `ls --flat` hides. `TestStorePathNoDotSegment` locks it in. This removes the
only trigger incredigo controls and is backend-agnostic.
2. **Independent coverage cross-check (done, harness).** `lab-harness.sh` T8b counts the
on-disk `*.gpg` files under `imported/` — ground truth that does **not** go through
`ls --flat` — and asserts the gate sealed exactly that many. This is where an
independent count is cheap and reliable (`GOPASS_HOMEDIR` is known/controlled). It would
have failed loudly on the original 7-vs-10.
**Production gate — deliberately *not* changed.** Adding the same on-disk `.gpg` walk to
`rotate.Snapshot` would couple the safety-critical gate to gopass's fs-backend layout
(store-dir resolution varies by gopass version/mount and breaks for non-fs backends). That
trades one fragility for another and could make the gate *falsely* abort or pass. The
two-layer defense above (write-side invariant + harness cross-check) closes the class
without that coupling. If we later want an in-gate coverage assertion, the clean path is a
backend-agnostic enumeration primitive in `sink` (not an fs-layout assumption) — a separate
design task.
## ICM evaluation (does the methodology help here?)
**Recommendation: do not adopt ICM for the incredigo codebase now.** ICM
(`icm-scaffold` skill) structures an *LLM-orchestrated* workflow as numbered
markdown stages — "folders are the program, one agent reads the right file at the
right moment." incredigo is a deterministic **compiled Go tool** with a package
architecture, an interface/registry extension model, and a real test suite. ICM's
folders-as-program model does not map onto a binary, and the project directive
explicitly bans an LLM in the destructive hot path. Adopting ICM here would add
scaffolding with no control-flow benefit. The one place it *could* fit later is the
**AI-router / browser long-tail** rotation layer (ROTATION.md step 4), which is by
design a skill/MCP layer *on top* of the Go core — revisit ICM only when that layer
is built.