Skip to content

Kernel VM Verification

pwnkit ingest --verify can run C reproducers inside a local QEMU guest and compare the guest dmesg output with the imported kernel crash report. Without this VM, kernel verification remains static-only and pwnkit will not claim that a crash was reproduced.

pwnkit ships a maintained build recipe at packages/core/src/triage/kernel-vm/. The recipe builds:

  • bzImage: Linux 6.8.12 for x86_64 with KASAN, UBSAN, KCSAN, lock debugging, RCU stall detection, virtio, 9p, ext4, NFS/NFSd, Bluetooth, WiFi, and SCTP support enabled.
  • rootfs.img: a 512 MB Debian Bookworm ext4 image with gcc, binutils, make, procps, kmod, strace, gdb, OpenSSH, and /sbin/pwnkit-init.
  • kernel.config: the exact kernel config used for the build.
  • pwnkit_vm_key and pwnkit_vm_key.pub: a root SSH keypair for manual guest debugging. The current pwnkit verifier does not use SSH; it communicates through a QEMU 9p shared directory.

The repository does not commit prebuilt VM artifacts. Build them locally or let the GitHub Actions E2E workflow build and cache them.

Install these host tools before building or running VM-backed verification:

  • Docker for the reproducible guest build.
  • QEMU, usually qemu-system-x86_64, for local verification runs.
  • At least 20 GB of free disk space for the Docker build cache and kernel build.
  • Enough memory for the guest. The runner defaults to 2048 MB.
  • Optional KVM acceleration on Linux. macOS and CI can run without acceleration, but boot and reproducer timeouts may need to be higher.

From the repository root:

Terminal window
pnpm install --frozen-lockfile
cd packages/core/src/triage/kernel-vm
PWNKIT_KERNEL_VM_MAKE_JOBS=4 ./build.sh "$HOME/.pwnkit/kernel-vm/linux-6.8.12-kasan"

Expected output directory:

$HOME/.pwnkit/kernel-vm/linux-6.8.12-kasan/
bzImage
rootfs.img
kernel.config
pwnkit_vm_key
pwnkit_vm_key.pub

The Dockerfile pins the kernel source to linux-6.8.12 from cdn.kernel.org and builds an amd64 Debian Bookworm guest. Treat the output directory as a local cache: regenerate it when the Dockerfile, kernel version, or guest package list changes.

Set the required environment variables:

Terminal window
export PWNKIT_KERNEL_QEMU=1
export PWNKIT_KERNEL_QEMU_KERNEL="$HOME/.pwnkit/kernel-vm/linux-6.8.12-kasan/bzImage"
export PWNKIT_KERNEL_QEMU_DISK="$HOME/.pwnkit/kernel-vm/linux-6.8.12-kasan/rootfs.img"

Recommended local defaults:

Terminal window
export PWNKIT_KERNEL_QEMU_MEMORY_MB=2048
export PWNKIT_KERNEL_QEMU_SMP=2
export PWNKIT_KERNEL_QEMU_BOOT_TIMEOUT_SEC=180
export PWNKIT_KERNEL_QEMU_TIMEOUT_SEC=60
export PWNKIT_KERNEL_QEMU_ARTIFACT_DIR="$HOME/.pwnkit/kernel-vm/runs"

On Linux hosts with KVM:

Terminal window
export PWNKIT_KERNEL_QEMU_ACCEL=kvm

Leave PWNKIT_KERNEL_QEMU_APPEND unset unless you use a custom guest. The default is:

console=ttyS0 root=/dev/vda rw nokaslr panic=-1 init=/sbin/pwnkit-init

Place syzbot-style crash reports and reproducers in the same directory. File stems are matched:

crashes/
bug-001.log
bug-001.c
bug-002.report
bug-002.syz

Run:

Terminal window
pwnkit ingest ./crashes --verify --output json

For each C reproducer, pwnkit:

  1. Creates a temporary host directory.
  2. Writes repro.c and runner.sh into that directory.
  3. Boots QEMU with bzImage, rootfs.img, and a 9p share mounted as pwnkitshare.
  4. Lets /sbin/pwnkit-init mount the share and execute /mnt/pwnkit/runner.sh.
  5. Compiles the reproducer with guest gcc.
  6. Runs the reproducer under the configured timeout.
  7. Copies compile.log, run.log, dmesg.log, marker files, and the serial log back to the host artifact directory when configured.

If PWNKIT_KERNEL_QEMU_ARTIFACT_DIR is unset, pwnkit deletes the temporary run directory after each verification attempt.

A custom guest must satisfy this contract:

RequirementContract
Architecturex86_64 guest bootable by qemu-system-x86_64
Root deviceroot=/dev/vda by default, or matching custom append line
Init path/sbin/pwnkit-init, unless PWNKIT_KERNEL_QEMU_APPEND is changed
Host shareMount QEMU 9p tag pwnkitshare at /mnt/pwnkit
RunnerExecute /mnt/pwnkit/runner.sh and leave results in the same share
Compiler/usr/bin/gcc plus libc headers and binutils
Logsdmesg readable after the reproducer runs
KernelDebug-friendly kernel with crash signal visible in dmesg

SSH is not part of the verifier contract. The generated keypair is exported only for manual debugging if you boot the image yourself with a network device and port forwarding.

VariableRequiredDefaultDescription
PWNKIT_KERNEL_QEMUYes-Set to 1 to enable VM execution.
PWNKIT_KERNEL_QEMU_KERNELYes-Path to bzImage.
PWNKIT_KERNEL_QEMU_DISKYes-Path to rootfs.img or another bootable disk.
PWNKIT_KERNEL_QEMU_BINARYNoqemu-system-x86_64QEMU binary to execute.
PWNKIT_KERNEL_QEMU_DISK_FORMATNoinferredraw or qcow2; inferred from extension when unset.
PWNKIT_KERNEL_QEMU_MEMORY_MBNo2048Guest memory in MB.
PWNKIT_KERNEL_QEMU_SMPNo2Guest CPU count.
PWNKIT_KERNEL_QEMU_APPENDNosee aboveKernel command line.
PWNKIT_KERNEL_QEMU_ACCELNo-QEMU accelerator, for example kvm.
PWNKIT_KERNEL_QEMU_INITRDNo-Optional initrd path for custom guests.
PWNKIT_KERNEL_QEMU_BOOT_TIMEOUT_SECNo120Time allowed for boot and guest setup.
PWNKIT_KERNEL_QEMU_TIMEOUT_SECNo60Time allowed for the reproducer.
PWNKIT_KERNEL_QEMU_SHARE_TAGNopwnkitshare9p mount tag expected by the guest init.
PWNKIT_KERNEL_QEMU_ARTIFACT_DIRNo-Host directory where per-run artifacts are preserved.

If the VM exits before producing results, inspect serial.log in PWNKIT_KERNEL_QEMU_ARTIFACT_DIR. Common causes:

  • The guest did not mount the 9p share. Keep PWNKIT_KERNEL_QEMU_SHARE_TAG and /sbin/pwnkit-init in sync.
  • gcc or libc headers are missing in a custom guest.
  • The guest cannot read dmesg.
  • The boot timeout is too low on hosts without KVM.
  • A custom kernel append line no longer points at the correct root disk or init.

The repository E2E workflow, .github/workflows/kernel-validator-e2e.yml, is the smoke-tested reference for CI. It builds the same artifacts, boots QEMU, runs a real ingest --verify flow, and uploads the VM logs plus runner outputs as artifacts.

Maintainers can run .github/workflows/kernel-validator-batch.yml manually from GitHub Actions to validate a curated syzbot corpus against the real VM runner. The default corpus lives at scripts/kernel-validator-batch-corpus.json; the workflow also accepts a JSON override for one-off corpus experiments.

The workflow uploads summary.json, summary.md, each case’s result.json, raw CLI output, and per-case VM artifacts. summary.json separates VM-reproduced verdicts from static-only verdicts with these top-level counts: verified, reproduced, crashMatch, reproducedMismatch, staticOnly, failed, and errored.

The batch workflow is workflow_dispatch only. Add a schedule only after the curated corpus and VM artifact cache are stable enough for unattended runs.