The Swap Storm: VMEXIT_INTR Infinite Loop

Stage 9 of my hypervisor runs a guest VM inside the blue-pill by swapping VMCBs. The host has its own VMCB. The guest has its own VMCB. On a signal, I save the host VMCB and load the guest VMCB. The guest runs until it exits, then I swap back. Simple concept. What actually happened was an infinite ping-pong that froze the entire machine in under a second.

The system didn't BSOD. It didn't black-screen. It just stopped responding. Mouse frozen, keyboard dead, no crash dump. That's the worst kind of failure because there's nothing to analyze after the fact. I had to reconstruct what happened from exit traces captured before the freeze.

The VMCB swap mechanism

The idea behind VMCB swap is straightforward. The host OS is already running as a guest under my blue-pill. It has a VMCB. When I want to run a second guest (the actual VM), I need to switch the CPU from executing under the host's VMCB to executing under the guest's VMCB.

// Simplified swap logic in the VMEXIT handler:
if per_cpu.switch_to_guest {
    // save host context
    vmsave(per_cpu.host_vmcb_pa);
    // load guest context
    vmload(per_cpu.guest_vmcb_pa);
    // point the vmrun loop at the guest VMCB
    per_cpu.active_vmcb = per_cpu.guest_vmcb_pa;
    per_cpu.is_guest_vm = true;
    per_cpu.switch_to_guest = false;
}

// On guest VMEXIT, check if we need to swap back:
if per_cpu.is_guest_vm && exit_code == VMEXIT_INTR {
    // guest yielded due to interrupt — swap back to host
    vmsave(per_cpu.guest_vmcb_pa);
    vmload(per_cpu.host_vmcb_pa);
    per_cpu.active_vmcb = per_cpu.host_vmcb_pa;
    per_cpu.is_guest_vm = false;
    per_cpu.switch_to_guest = true;
}

The storm

Here's what happens with that last line. The guest exits on VMEXIT_INTR because a clock interrupt fired. I swap to the host VMCB and set switch_to_guest = true. The host resumes, executes maybe one or two instructions, and then its next VMEXIT fires -- often also VMEXIT_INTR because there are more pending interrupts. My handler sees switch_to_guest = true and immediately swaps back to the guest.

The guest runs briefly, gets another interrupt, exits, swaps to host. Host gets an interrupt, swaps to guest. Guest gets an interrupt, swaps to host. Infinite loop. The core spends 100% of its time swapping VMCBs and never actually processes any host interrupts.

// The swap storm timeline:
//
// t0:  guest VMRUN
// t1:  guest VMEXIT_INTR → swap to host, re-arm
// t2:  host VMRUN
// t3:  host VMEXIT (any) → switch_to_guest=true → swap to guest
// t4:  guest VMRUN
// t5:  guest VMEXIT_INTR → swap to host, re-arm
// t6:  host VMRUN
// t7:  host VMEXIT → swap to guest
//      ... forever ...
//
// Host OS on this core STARVES.
// If this is core 0 (BSP), the ENTIRE system freezes.
host_exits_before_swap: 1-2
guest_exits_before_yield: 1

The fix: exit countdown

The solution is to give the host time to breathe. After the guest yields, instead of immediately re-arming switch_to_guest, I set a countdown. The host has to process 50 exits before I'll consider swapping back to the guest.

// Fixed swap-back logic:
if per_cpu.is_guest_vm && exit_code == VMEXIT_INTR {
    vmsave(per_cpu.guest_vmcb_pa);
    vmload(per_cpu.host_vmcb_pa);
    per_cpu.active_vmcb = per_cpu.host_vmcb_pa;
    per_cpu.is_guest_vm = false;
    // DON'T re-arm immediately. Give host 50 exits first.
    per_cpu.host_exit_countdown = 50;
}

// In the main exit handler, decrement:
if per_cpu.host_exit_countdown > 0 {
    per_cpu.host_exit_countdown -= 1;
    if per_cpu.host_exit_countdown == 0 {
        per_cpu.switch_to_guest = true;
    }
}

50 is a tuned value. Too low and the storm still happens under heavy interrupt load. Too high and the guest gets terrible latency. At 50, the host processes its clock interrupts, DPCs, and any pending I/O before the guest gets another timeslice. The system went from "instant freeze" to "2.7 million exits with OVMF booting to the DXE phase."

Why it was hidden behind the register bug

This is the cruelest part. The swap storm was there from the very first VMCB swap implementation. But I didn't see it because the GuestRegs/GuestGprs layout bug was also present. With scrambled registers, the guest would triple-fault within a handful of exits. The guest died so fast that the swap storm never had time to develop.

When I fixed the register layout, the guest suddenly lived long enough to hit clock interrupts and start the ping-pong cycle. I thought I'd broken something by fixing the registers. In reality I'd just revealed the second bug that was always there, hiding behind the first one.

// The debugging timeline:
//
// Day 1: GuestRegs cast bug present
//   → guest triple-faults in <10 exits
//   → guest dies → no swap storm
//   → system appears "stable" (guest just doesn't work)
//
// Day 2: Fixed register layout
//   → guest lives → hits VMEXIT_INTR → swap storm
//   → system FREEZES
//   → "I broke it by fixing the registers??"
//
// Day 2 evening: Found the re-arm bug
//   → exit countdown = 50
//   → 2.7M exits, OVMF booting, system responsive

Layered bugs

The VMCB swap had five distinct bugs that I found in sequence: the GuestRegs layout mismatch, host stealth post-processing running on guest exits, rdmsr faulting at GIF=0 for unknown MSRs, NPT allocation deadlocking at GIF=0 because the kernel allocator needs interrupts, and finally the swap storm itself.

Each bug masked the next. Fixing one revealed the next. The system went from "guest doesn't boot" to "guest boots but system freezes" to "everything works." The swap storm was the last one -- the one that was actually causing the freeze the whole time, hidden behind four layers of earlier failures.