VirtualizeCore(): Hardcoded for Windows, Broken for UEFI

I wrote VirtualizeCore() in entry.c with hardcoded values: CR4=0x3706E8, EFER=0xD01, CS=0x10, SS=0x18. These are correct for a running Windows kernel. They are completely wrong for UEFI. If the timer fires during UEFI boot, the VMCB gets wrong state and the machine crashes.

I discovered this the hard way. After fixing the timer threshold issue (post 30), I lowered the threshold to test earlier detection. The timer fired during UEFI DXE phase instead of during Windows kernel execution. VirtualizeCore() built a VMCB with Windows kernel state, called VMRUN, and the CPU resumed with incorrect CR4 and segment registers. Instant crash. No serial output, no blue screen, just a hard reset.

The hardcoded values

Here's what the original code looked like. Every value was a constant I'd measured from a running Windows kernel and pasted in:

static void VirtualizeCore(void) {
    VMCB *vmcb = (VMCB *)gVmcbPage;

    // guest state -- all hardcoded from a running Windows kernel
    vmcb->save.cr4   = 0x3706E8;
    vmcb->save.efer  = 0xD01;
    vmcb->save.cs.selector  = 0x10;
    vmcb->save.cs.attrib    = 0x029B;
    vmcb->save.ss.selector  = 0x18;
    vmcb->save.ss.attrib    = 0x0093;
    vmcb->save.ds.selector  = 0x2B;
    vmcb->save.es.selector  = 0x2B;

    // these came from the interrupted context at least
    vmcb->save.rip  = ReadSaveStateRip();
    vmcb->save.rsp  = ReadSaveStateRsp();
    vmcb->save.cr3  = ReadSaveStateCr3();
    vmcb->save.rflags = ReadSaveStateRflags();

    // ... NPT setup, host save area, VMRUN
    DoVmrun(vmcb);
}

RIP, RSP, CR3, and RFLAGS were read from the SMM save state -- the interrupted context. Those are correct because they reflect where the CPU actually was. But CR4, EFER, and the segment registers? Hardcoded. And those change dramatically between Windows and UEFI.

UEFI vs Windows: how the state differs

UEFI and the Windows kernel run in completely different CPU configurations. They're both in 64-bit long mode, but that's about where the similarity ends.

// state comparison: UEFI DXE phase vs Windows kernel

// CR4
UEFI CR4:     0x000020   // PAE only
Windows CR4:  0x3706E8  // PAE+PSE+PGE+FXSR+XSAVE+FSGSBASE+...

// EFER
UEFI EFER:    0x500     // LME + LMA
Windows EFER: 0xD01     // LME + LMA + SCE + NXE + SVME

// CS selector
UEFI CS:      0x38
Windows CS:   0x10

// SS selector
UEFI SS:      0x30
Windows SS:   0x18

If I build a VMCB with CR4=0x3706E8 but the CPU was actually running with CR4=0x000020, the guest resumes execution with a CR4 that doesn't match the page tables it was using. CR4.PGE controls global page behavior. CR4.OSFXSR controls SSE state saves. CR4.OSXSAVE controls XSAVE/XRSTOR. If these don't match the running code's expectations, anything from a #UD to a silent data corruption can happen.

The segment selectors are even more dangerous. If I set CS to 0x10 but the UEFI GDT has a different descriptor at index 2, the CPU tries to execute code using a descriptor that might be a data segment, or might have the wrong base, or might not even be present. That's an immediate #GP or double fault.

The fix: read actual state from SMM save state

The SMM save state area contains the actual CPU state at the moment the SMI fired. This includes CR4, EFER, and all segment registers. I should have been reading these from the start instead of hardcoding them.

static void VirtualizeCore(void) {
    VMCB *vmcb = (VMCB *)gVmcbPage;

    // read ALL state from the SMM save state -- no hardcoding
    vmcb->save.cr4   = ReadSaveStateCr4();
    vmcb->save.efer  = ReadSaveStateEfer() | 0x1000; // set SVME
    vmcb->save.rip   = ReadSaveStateRip();
    vmcb->save.rsp   = ReadSaveStateRsp();
    vmcb->save.cr3   = ReadSaveStateCr3();
    vmcb->save.cr0   = ReadSaveStateCr0();
    vmcb->save.rflags = ReadSaveStateRflags();

    // segment registers from save state
    vmcb->save.cs = ReadSaveStateCs();
    vmcb->save.ss = ReadSaveStateSs();
    vmcb->save.ds = ReadSaveStateDs();
    vmcb->save.es = ReadSaveStateEs();
    vmcb->save.fs = ReadSaveStateFs();
    vmcb->save.gs = ReadSaveStateGs();

    // GDT and IDT from save state
    vmcb->save.gdtr = ReadSaveStateGdtr();
    vmcb->save.idtr = ReadSaveStateIdtr();

    // ... NPT setup, host save area, VMRUN
    DoVmrun(vmcb);
}

Now VirtualizeCore() works regardless of when the SMI fires. If it fires during UEFI, the VMCB gets UEFI state. If it fires during the Windows kernel, the VMCB gets Windows state. The guest resumes with exactly the same CPU configuration it was interrupted with. The only thing I add is SVME in EFER (bit 12), because I need SVM enabled for VMRUN.

Reading the SMM save state

The SMM save state area is a CPU-defined data structure at a fixed offset within SMRAM. On AMD64, it's at SMBASE + 0xFE00 and extends to SMBASE + 0xFFFF. Each register has a specific offset. I wrote accessor functions for each one:

// AMD64 SMM save state offsets (from AMD APM Vol 2, Table 10-2)
#define SMM_SAVE_STATE_BASE  (gSmbase + 0xFE00)

#define SMM_RAX_OFFSET   0x1F8
#define SMM_RIP_OFFSET   0x1F8   // wait, that's wrong...

// actually, I used the UEFI SmmCpuIo protocol:
EFI_STATUS ReadSaveStateCr4(void) {
    UINT64 value;
    gSmmCpu->ReadSaveState(
        gSmmCpu,
        sizeof(UINT64),
        EFI_SMM_SAVE_STATE_REGISTER_CR4,
        gCurrentCpu,
        &value
    );
    return value;
}

Why I hardcoded in the first place

Laziness, honestly. When I first got SmmInfect working, the timer threshold was high enough that the kernel was always running when the callback fired. I dumped the state from one run, saw the register values, and pasted them in as constants. It worked. It passed my tests. So I shipped it and moved on to the next feature.

The problem surfaced when I changed the timing -- lowering the threshold to catch the kernel earlier. Suddenly the callback fired during UEFI, and the hardcoded values didn't match. This is the classic trap of hardcoding: it works in the scenario you tested and explodes in every other scenario.

The fix took about an hour -- writing the save state reader functions and replacing every hardcoded value. Testing it took another hour, verifying that the VMCB state was correct at different points in the boot. Two hours of work that I should have done from the beginning. The three days I spent debugging the timing issue (post 30) would have been avoided entirely if VirtualizeCore() had been correct from the start.