VMCB Swap: Running a Guest VM Inside a Blue-Pill

Stage 9 -- I run OVMF as a guest VM inside the blue-pill, on dedicated physical cores. The Windows host keeps running on cores 0-13 without knowing anything changed. Cores 14 and 15 belong to my guest. Two VMCBs per core, hot-swappable. OVMF boots to the BdsDxe menu with 2 virtual CPUs.

This is the point where the project stopped being "just" a blue-pill and became something closer to a real hypervisor. I'm managing two separate execution contexts on the same physical hardware -- the host OS that thinks it's running bare-metal, and a guest VM that thinks it's the only thing running. Neither knows about the other.

Dedicated cores: GUEST_CORE_BASE = 14

I designate physical cores 14 and 15 for the guest VM. GUEST_CORE_BASE = 14, so vCPU N maps to physical core 14 + N. Windows still has cores 0-13 -- all 14 logical processors. The guest gets 2.

Virtual APIC ID spoofing makes the guest see APIC IDs 0 and 1 instead of 14 and 15. This matters because OVMF (and any OS) uses APIC IDs to enumerate processors. If it saw IDs 14/15, it would think 14 CPUs were missing and might refuse to boot or behave erratically. I intercept the APIC base MSR reads and CPUID topology leaves to present a clean 2-processor view.

KVM-style atomic SIPI

When OVMF boots, the BSP (Bootstrap Processor, vCPU 0) sends a SIPI (Startup Inter-Processor Interrupt) to wake the AP (Application Processor, vCPU 1). In real hardware, SIPI is delivered through the APIC. In my setup, I intercept it and use atomic flags.

// BSP sends SIPI -- I intercept the APIC write
fn handle_sipi(target_apic_id: u8, vector: u8) {
    // store the SIPI vector for the target AP
    SIPI_VECTOR[target_apic_id]
        .store(vector, Ordering::Release);
    SIPI_PENDING[target_apic_id]
        .store(true, Ordering::Release);
}

// AP is halted, waiting for SIPI
fn handle_hlt(core_id: usize, vmcb: &mut VMCB) {
    if SIPI_PENDING[core_id].load(Ordering::Acquire) {
        let vec = SIPI_VECTOR[core_id].load(Ordering::Acquire);
        // real-mode reset: CS = vector << 8, IP = 0
        vmcb.save.cs.base = (vec as u64) << 12;
        vmcb.save.cs.selector = (vec as u16) << 8;
        vmcb.save.rip = 0;
        SIPI_PENDING[core_id].store(false, Ordering::Release);
    }
}

The swap storm

This was the worst bug in Stage 9. When the guest VM yields (say, it executes HLT or hits a VMEXIT_INTR), I need to switch back to the host VMCB so Windows can run on that core again. Simple enough -- swap the VMCB pointer and VMRUN with the host context.

The problem: the host would immediately hit an intercept (timer interrupt, typically), which would trigger a check: "is the guest VM ready?" Yes, it is -- the INTR was handled. So it swaps back to the guest. The guest hits another INTR within microseconds. Back to the host. Back to the guest. Thousands of swaps per second. Both VMCBs thrashing, neither making progress.

// THE FIX: cooldown counter
fn handle_guest_exit(data: &mut SvmData) {
    match data.guest_exit_code {
        VMEXIT_INTR => {
            // don't immediately switch back -- let host breathe
            data.switch_to_guest = false;
            data.host_exit_countdown = 50;
        }
        // ...
    }
}

// in the host exit handler:
if data.host_exit_countdown > 0 {
    data.host_exit_countdown -= 1;
} else if data.guest_armed {
    data.switch_to_guest = true;
}

The fix is a cooldown. When the guest yields, I set host_exit_countdown = 50. The host has to process 50 of its own exits before the guest becomes eligible again. This gives Windows time to handle its pending interrupts and DPCs without being yanked back into the guest context. The number 50 was empirical -- too low and the storm returns, too high and the guest starves.

GIF=0 safety

Guest exit handlers run at GIF=0 -- Global Interrupt Flag disabled. This is the SimpleSvm pattern. At GIF=0, no interrupts can fire, no NMIs, no SMIs. The CPU is completely locked to my handler code. This sounds great for atomicity, but it means I cannot do anything that might block or allocate.

Specifically: rdmsr for unknown MSRs returns 0 (I can't risk a #GP at GIF=0). NPF handling uses try_map_4k_noalloc -- a version of the page mapper that returns an error instead of calling ExAllocatePool. No kernel allocator calls. No waiting on locks. No IPI delivery. Everything that runs in a guest exit handler must be wait-free and allocation-free.

GuestRegs vs GuestGprs

This bug cost me an entire day. The blue-pill assembly saves registers in r15..rcx order (GuestRegs layout). The Stage 8 guest VM assembly saves them in rcx..r15 order (GuestGprs layout). Both are arrays of 16 u64 values. Both contain the same data. But the field order is reversed.

I had a handler that cast a GuestRegs pointer to GuestGprs because "they're the same struct, right?" No. GuestRegs.rax is at offset 0, GuestGprs.rax is at offset 120. Every register read was returning the wrong value. The guest VM was executing with garbage in every GPR. It took WinDbg + single-stepping the assembly to figure out why OVMF was dereferencing null pointers immediately after VMRUN.

Two VMs, one machine

With all of this working, I have OVMF booting to the BdsDxe menu on cores 14/15. It sees 2 processors, 2GB of RAM (carved from the host's physical memory via NPT), and an NVMe device I emulate through MMIO intercepts. Windows runs on cores 0-13 like nothing happened. Task Manager shows all 16 cores, but two of them show near-zero utilization because they're actually running my guest VM.

Stage 9 was the foundation for everything that came next -- booting actual Windows as a guest, GPU passthrough experiments, the whole VMM roadmap. But the lessons from the swap storm and the GIF=0 constraints shaped every design decision going forward.