SKELETONKEY/tools/verify-vm/README.md

SKELETONKEY VM verification

Auto-provisions a Parallels Desktop VM with a known-vulnerable kernel, runs skeletonkey --explain <module> --active inside it, and emits a verification record. Closes the loop between "detect() compiles & passes unit tests" and "exploit() actually works on a real vulnerable kernel."

One-time setup

./tools/verify-vm/setup.sh

That installs (if missing): Vagrant via Homebrew, the vagrant-parallels plugin, and pre-downloads ~5 GB of base boxes (Ubuntu 18.04/20.04/22.04

• Debian 11/12). Idempotent — re-run any time.

To skip boxes you don't need (save disk):

./tools/verify-vm/setup.sh ubuntu2004 debian11   # only those two

Verify a single module

./tools/verify-vm/verify.sh nf_tables

What that does (two-phase model — install kernel, then verify):

1. Reads tools/verify-vm/targets.yaml: finds nf_tables → box generic/ubuntu2204 + mainline_version: 5.15.5.
2. vagrant up skk-nf_tables if not already running (each module gets its own machine for isolation).
3. Prep phase — runs every prep provisioner that applies:
   • pin-kernel-<pkg> if kernel_pkg is set (apt install + GRUB_DEFAULT pin)
   • pin-mainline-<ver> if mainline_version is set (download from kernel.ubuntu.com/mainline, dpkg -i, GRUB_DEFAULT pin)
   • module-provision-<name> if provisioners/<name>.sh exists (build vulnerable sudo from source, drop polkit allow rule, install udisks2, etc.)
4. Conditional reboot — vagrant reload if uname -r doesn't match the target kernel after the prep phase. Confirms post-reboot kernel actually landed on the target; warns if it didn't.
5. Verify phase — build-and-verify provisioner: rsync the source, make, run skeletonkey --explain <module> --active.
6. Parses the VERDICT: line, compares against expect_detect from targets.yaml, appends a JSON verification record to docs/VERIFICATIONS.jsonl.
7. Suspends the VM (vagrant suspend) — instant resume next run.

Lifecycle flags:

./tools/verify-vm/verify.sh nf_tables --keep      # leave VM running; ssh in to inspect
./tools/verify-vm/verify.sh nf_tables --destroy   # full teardown after run

List every target

./tools/verify-vm/verify.sh --list

Shows the (module, box, target kernel, expected verdict, notes) matrix for all targets. Modules with manual: true are blocked by their target environment — see the notes field for the reason (VMware-only guest, EOL kernel needed, t64-transition libs missing, etc.).

Verification records

verify.sh appends one JSON record per run to docs/VERIFICATIONS.jsonl:

{
  "module":        "nf_tables",
  "verified_at":   "2026-05-24T03:24:01Z",
  "host_kernel":   "5.15.5-051505-generic",
  "host_distro":   "Ubuntu 22.04.3 LTS",
  "vm_box":        "generic/ubuntu2204",
  "expect_detect": "VULNERABLE",
  "actual_detect": "VULNERABLE",
  "status":        "match"
}

status: match means detect() returned what we expected on a known- vulnerable kernel. Anything else (MISMATCH, exit code != 0) means either:

• The kernel pin didn't take — check host_kernel against kernel_version in targets.yaml. The "post-reboot kernel" line in the verify log will say if vagrant reload did or didn't land on the target.
• The exploit's preconditions aren't met in the default Vagrant image (e.g. apparmor blocks unprivileged userns; provisioner needed).
• The module's detect() logic is wrong for this kernel/distro combo (a real module bug — fix it, as we did for dirtydecrypt after cross-checking against NVD).

Run tools/refresh-verifications.py after new records land to regenerate core/verifications.c so the binary's --explain and --list reflect the latest evidence.

How it routes module → box

Mapping lives in tools/verify-vm/targets.yaml. Each entry has:

• box — generic/ (e.g. ubuntu2204)
• kernel_pkg — apt package for a vulnerable stock-archive kernel, if one still exists in the distro's repos
• mainline_version — alternative to kernel_pkg: pulls a vanilla upstream kernel from kernel.ubuntu.com/mainline/v<ver>/. Use when the apt-archive version has been garbage-collected (Ubuntu drops old ABI versions) or when you need a specific point release that the distro never packaged.
• kernel_version — what uname -r should report after install
• expect_detect — VULNERABLE | OK | PRECOND_FAIL
• manual: true — skip auto verification; explain why in notes
• notes — full context for why this target was picked

Adding a new module is one block in targets.yaml. If the module needs per-target setup beyond installing a kernel — for example building sudo from source, adding a sudoers grant, or dropping a polkit allow rule — write a shell script at tools/verify-vm/provisioners/<module>.sh and the Vagrantfile will pick it up automatically.

Module-specific provisioners (provisioners/<module>.sh)

When the kernel pin alone doesn't make a host vulnerable — e.g. the bug is sudo-version-gated, or a polkit "active session" check blocks the SSH path — drop a shell script at tools/verify-vm/provisioners/<module_name>.sh. The Vagrantfile runs it as root in the prep phase, before the vagrant reload check. Scripts should be idempotent (apt is no-op if installed, file overwrites are safe) since they re-run on every verify.

Existing examples:

• sudo_chwoot.sh — builds sudo 1.9.16p1 from upstream into /usr/local/bin so the vulnerable --chroot code path is reachable on Ubuntu 22.04 (which ships pre-feature 1.9.9).
• udisks_libblockdev.sh — installs udisks2 + drops a polkit rule allowing the vagrant user to invoke loop-setup / filesystem-mount (without this, the SSH session is not "active" per polkit and the D-Bus call short-circuits).
• sudo_runas_neg1.sh — adds vagrant ALL=(ALL,!root) NOPASSWD: /bin/vi to /etc/sudoers.d/ so find_runas_blacklist_grant() has a grant to abuse.

Pinning kernels: apt vs mainline

pin-kernel-<pkg> runs apt-get install -y <pkg>. Best when the target version still lives in the distro's archive (rare for old point releases — Ubuntu eventually GCs them). Also pins GRUB_DEFAULT to the just-installed kernel so the reboot lands on it instead of the higher-version stock kernel.

pin-mainline-<ver> downloads vanilla mainline debs from kernel.ubuntu.com/mainline/v<ver>/. Tries /amd64/ first, falls back to bare /v<ver>/ for old kernels (≤ ~4.15) where amd64 wasn't a separate subdir. Accepts both linux-image- (older naming) and linux-image-unsigned- (current). Pins GRUB_DEFAULT to the mainline kernel so grub doesn't keep booting the higher-versioned stock kernel.

Files

tools/verify-vm/
├── README.md         this file
├── setup.sh          one-time bootstrap (Vagrant, plugin, box cache)
├── verify.sh         per-module verifier
├── Vagrantfile       parameterized VM config (driven by SKK_VM_* env vars)
├── targets.yaml      module → box mapping with rationale
├── provisioners/     optional per-module shell hooks
│   ├── sudo_chwoot.sh
│   ├── sudo_runas_neg1.sh
│   └── udisks_libblockdev.sh
└── logs/             per-verification stdout/stderr capture

Why Vagrant + Parallels

You already have Parallels Desktop. vagrant-parallels gives a scriptable per-VM config + a curated public box library + idempotent vagrant up/provision/reload/suspend lifecycle. The Vagrantfile is parameterized via env vars so a single file drives every target.

Alternative providers (Lima, Multipass) would also work; Vagrant was chosen for ergonomic continuity with the existing Parallels install.