GDT Patching: Fixing #GP in the Boot Path

The Windows bootloader expects a specific GDT layout. My hypervisor's GDT didn't match. The CPU raised #GP(0) on a segment register load, the guest triple-faulted, and I spent three days figuring out why. The fix was simple -- the diagnosis was not.

This was the first wall I hit trying to boot Windows as a guest VM. OVMF had been booting fine for weeks -- it doesn't care about the GDT layout as long as the entries are valid. But winload.efi is picky. It builds its own GDT during the transition from UEFI to the Windows kernel, and the selectors it uses are hardcoded. If the GDT doesn't match, the CPU throws an exception and the boot dies instantly.

What the serial output showed

I had serial debugging enabled via BCD, so I could see winload's progress. The last message before the crash was the transition function -- OslArchTransferToKernel. This is the point where winload switches from UEFI's address space to the kernel's address space. It sets up a new GDT, loads new segment registers, switches CR3, and jumps to the kernel entry point.

My #VMEXIT handler caught the exception before the triple fault. The error code was 0 (null selector), the faulting RIP was inside winload, and the faulting instruction was mov ss, ax with AX = 0x18.

// #VMEXIT exception info from my handler log
exception: #GP(0)
rip:       0xFFFFF801'4E2A0042  // OslArchTransferToKernel+0x42
insn:      8E D0              // mov ss, ax
rax:       0x0000000000000018  // selector for GDT[3]
cs:        0x08               // loaded fine -- GDT[1] was code

Selector 0x18 means GDT index 3 (0x18 / 8 = 3), RPL 0 (ring 0). The CPU loads selector 0x18 into SS, looks up GDT entry 3, and checks whether it's a valid data segment. If it's a code segment, or if the descriptor is wrong in any other way, the CPU raises #GP.

What my GDT looked like vs what Windows expected

I dumped my guest's GDT from the VMCB and compared it to what Windows expects. The problem was immediately obvious.

// my GDT (inherited from OVMF/UEFI environment)
gdt[0] = 0x0000000000000000  // null
gdt[1] = 0x00AF9B000000FFFF  // code (0x08) -- CS works
gdt[2] = 0x00AF9B000000FFFF  // code (0x10) -- WRONG
gdt[3] = 0x00CF9B000000FFFF  // code (0x18) -- WRONG

// what Windows winload expects
gdt[0] = 0x0000000000000000  // null
gdt[1] = 0x00AF9B000000FFFF  // code (0x08) -- CS
gdt[2] = 0x00CF93000000FFFF  // data (0x10) -- DS
gdt[3] = 0x00CF93000000FFFF  // data (0x18) -- SS

The difference is in byte 5 of the descriptor -- the access byte. 0x9B means "code segment, execute/read, accessed." 0x93 means "data segment, read/write, accessed." My GDT entries 2 and 3 had code descriptors where Windows expected data descriptors. OVMF doesn't load SS from GDT[3], so it never hit this issue. But winload does, and the CPU won't let you load a code segment into SS.

The fix: intercept LGDT and patch

Winload issues an LGDT instruction to load its own GDT before doing the segment register loads. I intercept this in my #VMEXIT handler. When I see an LGDT from winload, I inspect the new GDT that's about to be loaded, check if entries 2 and 3 are data segments, and if not, patch them.

fn handle_lgdt(vmcb: &mut VMCB, gdt_base: u64, gdt_limit: u16) {
    // read the GDT entries the guest is about to load
    let entries = read_guest_gdt(gdt_base, gdt_limit);

    // check if entry 2 (DS, selector 0x10) is a data segment
    if !is_data_segment(entries[2]) {
        // patch it to a ring 0 read/write data segment
        write_guest_gdt_entry(gdt_base, 2, 0x00CF93000000FFFF);
    }

    // check if entry 3 (SS, selector 0x18) is a data segment
    if !is_data_segment(entries[3]) {
        write_guest_gdt_entry(gdt_base, 3, 0x00CF93000000FFFF);
    }

    // let the LGDT proceed with corrected entries
    vmcb.save.gdtr_base = gdt_base;
    vmcb.save.gdtr_limit = gdt_limit as u32;
}

Why it took three days

The fix is 20 lines of code. The diagnosis took three days because of the triple fault. When a #GP happens during a segment load, the CPU tries to deliver the exception. If the exception handler itself faults (because the GDT is broken and the CPU can't find a valid exception handler segment), that's a double fault. If the double fault handler faults, that's a triple fault, and the CPU resets.

My hypervisor's default behavior on triple fault was to kill the guest immediately. I had no logging for the intermediate exceptions. The first day, I just saw the guest die with no output. The second day, I added exception frame capture to the #VMEXIT handler -- before the triple fault could fire, I logged the original exception's error code, RIP, and instruction bytes. The third day, I decoded the instruction, traced the selector to GDT[3], dumped the GDT, and found the mismatch.

After this experience, I permanently added full exception logging to the guest's #VMEXIT path. Every exception now gets the full frame captured -- vector, error code, RIP, RSP, instruction bytes, and the first 8 stack values. Never again will I debug a triple fault blind.

The exception handler I built

The exception logging system I added after this incident captures everything I need on the first occurrence. When a guest exception fires that would lead to a double or triple fault, I grab the state before the cascade starts.

// exception frame capture -- runs before triple fault can fire
fn log_guest_exception(vmcb: &VMCB) {
    let vec  = vmcb.control.exit_int_info & 0xFF;
    let code = vmcb.control.exit_int_info_err;
    let rip  = vmcb.save.rip;
    let rsp  = vmcb.save.rsp;

    log!("GUEST EXCEPTION: vec={:#x} code={:#x}", vec, code);
    log!("  RIP={:#018x} RSP={:#018x}", rip, rsp);

    // dump instruction bytes at RIP
    let bytes = read_guest_bytes(rip, 16);
    log!("  insn: {:02x?}", bytes);

    // dump first 8 stack values
    for i in 0..8 {
        let val = read_guest_u64(rsp + i * 8);
        log!("  [RSP+{:#x}] = {:#018x}", i * 8, val);
    }
}

This logging paid for itself within the first week. The MTRR issue (next post) also manifested as a guest exception, and having the full frame immediately told me the faulting MSR number and calling context. Without the logging, I would have been back to guessing.

The bigger picture

GDT layout seems like a trivial detail, and in some sense it is -- every OS textbook covers segment descriptors. But in a hypervisor context, you inherit whatever GDT the firmware or previous boot stage left behind. If you don't actively manage it, the mismatch between what the firmware set up and what the OS expects will bite you at the worst possible moment -- during the transition from bootloader to kernel, when there's no error handler running yet.

The x86 GDT is a legacy artifact from the 80286. In long mode (64-bit), most of its fields are ignored -- the base and limit of code/data segments don't matter because segmentation is effectively flat. But the descriptor type still matters. The CPU still checks whether you're loading a code segment into SS or a data segment into CS. It still validates the present bit, the DPL, and the system/code-data flag. Forty years of backward compatibility, and one bit in a descriptor type field can halt a modern OS boot.

This was the first of several fixes needed to get Windows booting. After the GDT, the next wall was MTRR emulation -- the kernel's memory manager reads Memory Type Range Registers during early init, and my hypervisor was returning garbage. But that's the next post.