MSR Filtering: The Hypercall Page and Synthetic Timers

After CPUID tells Windows that Hyper-V is running, the kernel immediately starts reading and writing Hyper-V synthetic MSRs. These are model-specific registers that don't exist on bare metal -- they're part of Hyper-V's paravirtualization interface. If Windows writes to HV_X64_MSR_HYPERCALL and nothing happens, the kernel panics. Every MSR I advertised in CPUID leaf 3 needs a working handler behind it.

AMD SVM gives me an MSR permission bitmap (MSRPM) -- a 8KB bitmap where each MSR has two bits: one for read intercepts, one for write intercepts. If a bit is set, the corresponding RDMSR/WRMSR triggers a #VMEXIT and I handle it. If it's clear, the instruction executes natively at full speed.

The goal is to intercept only the MSRs I need to fake, and let everything else pass through. Every intercepted MSR costs a VMEXIT -- about 400-600 cycles minimum. That's measurable.

HV_X64_MSR_GUEST_OS_ID (0x40000000)

This is the first MSR Windows writes after identifying Hyper-V via CPUID. It writes a 64-bit value encoding the OS type, version, and build number. Real Hyper-V uses this for compatibility -- different Windows versions get different feature sets.

I just store whatever Windows writes. Nothing depends on the value -- I don't change behavior based on OS version. But Windows expects the write to succeed and reads it back later, so I need to hold the value.

The hypercall page (0x40000001)

This is the critical one. Windows writes a physical page address to HV_X64_MSR_HYPERCALL, and Hyper-V is supposed to fill that page with a tiny stub that the guest uses to invoke hypercalls. On real Hyper-V, the stub contains a VMCALL instruction (Intel) or VMMCALL (AMD). Windows then calls this stub to request services from the hypervisor.

// Handle write to HV_X64_MSR_HYPERCALL (0x40000001)
fn handle_hypercall_msr_write(value: u64) {
    let enable = value & 1;
    let gpa = (value >> 12) << 12; // page-aligned GPA

    if enable == 1 {
        // map the guest page and write a VMMCALL;RET stub
        let page = map_guest_page(gpa);
        // VMMCALL (0x0F 0x01 0xD9) + RET (0xC3)
        page[0] = 0x0F; // VMMCALL opcode byte 1
        page[1] = 0x01; // VMMCALL opcode byte 2
        page[2] = 0xD9; // VMMCALL opcode byte 3
        page[3] = 0xC3; // RET
        unmap_guest_page(page);
    }
    // store the value so reads return what was written
    store_hyperv_msr(0x40000001, value | enable);
}

When Windows calls the hypercall page, it executes VMMCALL which triggers a #VMEXIT to my handler. I check the hypercall input (in RCX) and return the appropriate response. Most hypercalls I just return HV_STATUS_INVALID_HYPERCALL_CODE -- Windows handles the failure gracefully and falls back. The few that matter (like HvFlushVirtualAddressSpace) I actually implement.

VP_INDEX, VP_RUNTIME, and the identity MSRs

Windows reads HV_X64_MSR_VP_INDEX (0x40000002) to get the current virtual processor index. I return the current core's APIC ID. HV_X64_MSR_VP_RUNTIME (0x40000010) returns how long this VP has been running in 100ns units -- I compute it from TSC divided by the frequency.

// Virtual processor MSRs
fn handle_hyperv_msr_read(msr: u32, per_cpu: &PerCpu) -> u64 {
    match msr {
        0x40000002 => per_cpu.apic_id as u64,
        0x40000010 => {
            let tsc = rdtsc();
            tsc * 10_000_000 / per_cpu.tsc_freq
        }
        // Reference TSC frequency -- Windows uses this for QueryPerformanceCounter
        0x40000022 => per_cpu.tsc_freq,
        // APIC frequency -- needed for synthetic timer calibration
        0x40000023 => per_cpu.apic_freq,
        // Reference TSC page (contains TSC calibration data)
        0x40000021 => per_cpu.ref_tsc_page,
        _ => 0,
    }
}

TSC_FREQUENCY (0x40000022) and APIC_FREQUENCY (0x40000023) are critical. Windows uses TSC_FREQUENCY for QueryPerformanceCounter calibration. If I return a value that doesn't match the real TSC frequency, every timing measurement in the OS diverges from reality. So I read the actual frequency from the hardware and pass it through.

TIME_REF_COUNT and REFERENCE_TSC

HV_X64_MSR_TIME_REF_COUNT (0x40000020) returns a monotonic 100ns counter. Windows reads this as an alternative to RDTSC for wall-clock time. I compute it from TSC the same way as VP_RUNTIME.

The reference TSC page (HV_X64_MSR_REFERENCE_TSC, 0x40000021) is more interesting. When Windows writes a GPA to this MSR, I'm supposed to map a shared page there containing TSC calibration data -- scale, offset, and a sequence counter. Windows reads this page from usermode (it's mapped at KUSER_SHARED_DATA + 0x3B8) to do fast, paravirtualized time queries without any syscall or VMEXIT.

SynIC MSRs

The Synthetic Interrupt Controller is Hyper-V's paravirtualized APIC. Windows uses it for inter-processor signaling and synthetic timer delivery. The key MSRs are:

• HV_X64_MSR_SCONTROL (0x40000080) -- SynIC enable/disable. I store the value and track the enable bit.
• HV_X64_MSR_SIEFP (0x40000082) -- Synthetic Interrupt Event Flags Page. Windows writes a GPA; I allocate and map it.
• HV_X64_MSR_SIMP (0x40000083) -- Synthetic Interrupt Message Page. Same pattern -- store the GPA.
• HV_X64_MSR_EOM (0x40000084) -- End Of Message. Write signals that Windows has processed a synthetic interrupt. I just acknowledge it.

In practice, the SynIC MSRs get written during boot and then mostly ignored. Windows writes the pages, enables the controller, and then uses the synthetic timer MSRs (which are the ones that actually fire frequently).

Synthetic timer MSRs

Hyper-V provides 4 synthetic timers per virtual processor. Each timer has a config MSR and a count MSR. Windows typically uses timer 0 for the scheduler tick -- it replaces the legacy APIC timer with a paravirtualized version that's cheaper (no VMEXIT on expiry because the hypervisor delivers it directly).

I intercept the timer config MSR writes (0x400000B0 through 0x400000B3), parse the timer period and delivery mode, and set up a real APIC timer behind the scenes to approximate the requested interval. When it fires, I inject a synthetic interrupt vector through the guest's IDT. It's not perfect -- there's jitter compared to real Hyper-V's direct injection -- but Windows doesn't notice because the scheduler is tolerant of timer jitter.

The whole synthetic MSR surface is about 30 registers. Each one gets a read handler and a write handler in my VMEXIT dispatcher. Most are trivial store/load operations. The hypercall page and reference TSC page are the only ones with real logic behind them. But every single one has to exist and behave correctly, or Windows BSODs during boot with increasingly cryptic stop codes.

The MSRPM bitmap

The MSR permission bitmap is an 8KB region pointed to by vmcb.control.msrpm_base_pa. It has 4 regions covering different MSR ranges. For each MSR, two bits: bit 0 for read intercept, bit 1 for write intercept. Setting both means every access to that MSR causes a VMEXIT.

I mark all Hyper-V synthetic MSRs (0x40000000-0x400000FF) for intercept. I also intercept EFER reads (to hide the SVME bit), VM_CR (to hide the R_INIT bit), and a handful of timing-related MSRs (APERF, MPERF, RAPL). Everything else passes through natively -- the guest reads and writes real hardware MSRs without any hypervisor involvement.

The total number of intercepted MSRs is about 50 out of thousands. Every other MSR operates at native speed. This is what keeps the performance overhead near zero -- only the MSRs that would reveal the hypervisor's existence get trapped.