Building an AMD SVM Blue-Pill Hypervisor from Scratch

I wanted to virtualize Windows from underneath itself. No reboot, no UEFI modification, just load a driver and suddenly the entire OS is running inside my hypervisor without knowing it. That's what a blue-pill is.

The whole thing is written in Rust, compiled as a Windows kernel driver. #![no_std], WDK crate, single .sys file. I load it with sc create, it virtualizes all 16 cores, and when I'm done I sc stop and everything goes back to normal. No trace.

Took me about three months to get here. This post is the technical breakdown of how it works and what I learned building it.

The VMCB is everything

On AMD, the hypervisor talks to the CPU through a 4KB page called the VMCB. I fill in what I want to intercept, what state the guest should have, and then call VMRUN. The CPU takes over from there.

// VMCB control area
vmcb.control.intercept_cpuid    = 1;
vmcb.control.intercept_msr      = 1;
vmcb.control.intercept_vmrun    = 1;
vmcb.control.npt_enable         = 1;
vmcb.control.npt_cr3            = npt_root_pa;

Virtualizing all 16 cores

If even one logical processor isn't virtualized, anything running on that core can see the inconsistency. So every core has to go under.

for core_id in 0..num_cores {
    pin_thread_to_core(core_id);
    let vmcb = allocate_vmcb();
    setup_vmcb(vmcb, current_state());
    vmrun(vmcb);
    // right here -- this line runs inside the hypervisor now
    // the core just became a guest and kept executing
}

I pin a DPC to each core, snapshot its entire state -- RIP, RSP, CR3, all the segment registers, everything -- copy that into a VMCB, and call VMRUN. The CPU transitions into guest mode and picks up executing at the exact same instruction. The OS doesn't notice because from its perspective nothing changed. Same registers, same memory, same instruction pointer. It just can't see my code anymore.

This was the first moment where it actually felt real. I run systeminfo and it says "A hypervisor has been detected." That's my hypervisor.

NPT -- hiding from the guest

Nested Page Tables give me a second layer of address translation that the guest can't see. The guest thinks physical address 0x1000 maps to actual physical address 0x1000, and for most of memory it does -- I build NPT as identity-mapped. But for the pages where my hypervisor code lives, my VMCBs, my data structures? Those entries point to nothing.

If the guest tries to read my memory, it triggers a nested page fault (#NPF). I handle that by returning zeros. The guest can scan all of physical memory and it'll never find me because those pages literally don't exist in its address space.

I spent a solid week getting NPT right. The page table structure itself isn't hard -- it's the same 4-level format as regular x86 paging. The hard part is making sure I don't accidentally unmap something the OS needs, and handling large pages (2MB) correctly when I need to split them to hide a single 4KB page.

TSC compensation

This is where it gets interesting. RDTSC measures CPU cycles, and every time the CPU transitions from guest to hypervisor (VMRUN) and back (#VMEXIT), it burns cycles. If something runs RDTSC before and after a CPUID instruction, and the gap is way bigger than it should be, it knows something is intercepting that CPUID. That's how anti-cheats detect hypervisors.

AMD gives a tsc_offset field in the VMCB. Whatever value I put there gets added to the guest's TSC reads. So I measure the average exit cost -- on my Ryzen 7 it's about 150 cycles -- and subtract it:

vmcb.control.tsc_offset = -(avg_exit_cost as i64);

It's not perfect. The exit cost varies by a few cycles each time, so statistical analysis over thousands of samples can still catch it. But it turns detection from "one RDTSC pair" into "collect and analyze timing distributions," which is a massive difference.

What it looks like running

After all this, I have a driver that loads in under 100ms, virtualizes all 16 logical processors, hides itself via NPT, compensates TSC timing, and presents itself as Microsoft Hyper-V via CPUID leaf 0x40000000. Windows thinks Hyper-V is running. VBS queries see what I want them to see.

The devirt path is clean too -- sc stop restores every core to bare metal. No BSOD, no leftover state, no trace in memory. I can virtualize and devirtualize repeatedly without rebooting.

Stage 7 was when this all came together. From here the project goes deeper -- spoofing Core Isolation, running guest VMs inside the blue-pill, flashing the hypervisor into firmware. But this is the foundation everything else is built on.