rotate: promote k8s + mongo to LIVE-VM via real-target POCs

Ran live cutover POCs in the sandbox VM against the real target software and
promoted both drivers from MOCK-ONLY to LIVE-VM in the proof table (data, not
driver behaviour):

- k8s: proven against real k3s v1.35 kube-apiserver + token controller
  (lab-provision-k8s.sh). The real apiserver serves a self-signed cert, so the
  driver gained CA-pinned TLS — blob params ca= (base64 PEM, pinned as the only
  trusted root) and insecure=1 (lab-only escape hatch), routed through a new
  clientFor(cr). Neither is test-awareness; a real deployment configures the same
  CA. Added TestK8sTLSTrust covering CA-pinned / insecure / untrusted-rejection
  and the blob round-trip.
- mongo: proven against real mongod 8.0 (lab-provision-mongo.sh).

npm stays MOCK-ONLY by decision (registry.npmjs.org not self-hostable; real
create-token requires the account password in the body) — documented as a
contract gap rather than faking a LIVE-VM. gcp/twilio/flyio remain MOCK-ONLY.

Live POCs surfaced real-target-only behaviour now recorded in the docs: the
apiserver's ~10s successful-auth cache (old token kept working ~7s after Secret
deletion) and real mongosh's stdout prompt. Full suite 104 green, -race clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
leetcrypt
2026-06-18 21:50:21 -07:00
parent 35f19c007a
commit 9109da7ae4
5 changed files with 160 additions and 35 deletions
+26 -10
View File
@@ -41,6 +41,17 @@ stops a mock-only driver from ever being mistaken for a live one.
| `wireguard` | real `wg` | `lab-provision-wg.sh` |
| `gitea` | real Gitea | `lab-provision-gitea.sh` |
| `appsecret` | real local files | `lab-provision-appsec.sh` |
| `mongo` | real `mongod` 8.0 | `lab-provision-mongo.sh` |
| `k8s` | real k3s `v1.35` kube-apiserver | `lab-provision-k8s.sh` |
> **k8s live-POC note (auth-cache timing):** the real kube-apiserver caches *successful*
> token authentications for ~10s (`cachedTokenAuthenticator`). After `RevokeOld` deletes the
> legacy token Secret, the old token keeps authenticating until that cache entry expires — so
> the POC polls the old token for up to 30s and asserts it flips to `401` (it does, ~7s). The
> `httptest` emulator invalidates instantly, so this delay only surfaced against real k3s —
> exactly what a live POC is for. The driver needed real-CA TLS to talk to the apiserver, so
> the blob gained `ca=` (base64 PEM, pinned as the only trusted root) and an `insecure=1`
> lab-only escape hatch; neither is test-awareness — a real deployment configures the same CA.
### MOCK-ONLY — proven only against an emulator / mock
@@ -53,12 +64,10 @@ stops a mock-only driver from ever being mistaken for a live one.
| `cloudflare` | `httptest` emulator | no self-host |
| `ghactions` | `httptest` emulator (holds the libsodium box keypair, decrypts the sealed PUT) | no self-host |
| `gitlab` | `httptest` emulator | GitLab CE too heavy for the VM |
| `mongo` | fake-`mongosh` shell stub (auths against a state file) | real-MongoDB VM POC not yet run |
| `npm` | `httptest` registry emulator | real registry never hit |
| `npm` | `httptest` registry emulator | real registry never hit (see caveat below) |
| `gcp` | `httptest` IAM emulator (verifies the driver's RS256 JWT against the key's public key) | no self-host |
| `twilio` | `httptest` emulator (Basic-auth Keys resource) | no self-host |
| `flyio` | `httptest` GraphQL emulator | no self-host |
| `k8s` | `httptest` apiserver emulator (token-Secret create/delete) | real k3s POC not yet run |
## What the MOCK-ONLY emulators actually prove
@@ -73,7 +82,7 @@ protocol so the test exercises the driver's true cutover logic:
on `PUT` **decrypts the sealed value** and stores the plaintext; the test asserts the
decrypted value equals the new value the driver put in the rebuilt blob — proving the
sealed-box encryption is correct, not simulated.
- **npm / twilio / k8s** — each emulator enforces credential validity (a token/key
- **npm / twilio** — each emulator enforces credential validity (a token/key
authenticates only while it exists) and the test asserts `Verify(old)` fails after
`RevokeOld` — proving the create→verify→revoke cutover really happened, old credential
dead, new one alive.
@@ -85,10 +94,17 @@ protocol so the test exercises the driver's true cutover logic:
- **flyio** — a managing token mints/deletes the deploy token over GraphQL; the deploy token
authenticates the app query only while it exists, so the test proves the managing-token →
deploy-token rotation and that the revoked deploy token stops working.
- **mongo** — the fake `mongosh` authenticates `db.auth` against a password held in a state
file and only `changeUserPassword` updates it; the test proves the in-place cutover (old
password rejected after rotation). No password ever reaches `mongosh`'s argv.
Promoting any of these to LIVE-VM only requires standing up the real service (a self-hosted
GitLab CE / a real Cloudflare token / a throwaway GitHub repo / a real MongoDB / a real npm
registry / a k3s cluster) and re-running the same driver against it — no driver code changes.
> **npm caveat (why it is not just "pending a VM run"):** the public `registry.npmjs.org`
> is not self-hostable, and the classic `POST /-/npm/v1/tokens` create-token call on real
> npm **requires the account password in the request body** (it is the non-interactive form
> of `npm token create`, which prompts for your password). The driver authenticates the
> create with the *old token* as Bearer only, which the emulator accepts but real npmjs does
> not. So `npm` stays MOCK-ONLY pending either (a) a self-hosted registry that implements the
> token API faithfully, or (b) adding password-in-body support to the create call. This is
> exactly the kind of contract gap a live-POC attempt is meant to surface.
Promoting the rest to LIVE-VM only requires standing up the real service (a self-hosted
GitLab CE / a real Cloudflare token / a throwaway GitHub repo) and re-running the same
driver against it — no driver code changes. (`k8s` and `mongo` have already made this jump;
see the LIVE-VM table above.)