.. /blog
2026-03-02 // 8 min

GIF=0: Running Exit Handlers with Interrupts Masked

gifsafetyamd-svm

When the CPU takes a #VMEXIT on AMD, it clears the Global Interrupt Flag. GIF=0 means absolutely nothing can interrupt my exit handler -- no external interrupts, no NMIs, no SMIs. This sounds like a feature. It's actually a minefield.

The SimpleSvm pattern -- the one most open-source AMD hypervisors follow -- runs the entire exit handler at GIF=0. No STGI before dispatch. I do the same thing. This post is about why that works, what it costs, and all the ways it almost killed my hypervisor.

What GIF=0 actually means

The Global Interrupt Flag is separate from RFLAGS.IF. IF controls whether the CPU delivers external interrupts to the running code. GIF controls whether the CPU even acknowledges they exist. At GIF=0, the CPU ignores everything -- INTR, NMI, SMI, INIT. They're queued but not delivered until GIF goes back to 1.

// timeline of a VMEXIT
//
// VMRUN (GIF=1, guest executing)
//   |
//   +-- intercept fires (CPUID, MSR, NPF, etc.)
//   |
// #VMEXIT
//   |-- hardware saves guest state to VMCB
//   |-- hardware loads host state from host save area
//   |-- hardware clears GIF (GIF=0)
//   |
//   +-- my exit handler runs HERE (GIF=0)
//   |     no interrupts, no NMI, no SMI
//   |     cannot call ANY blocking kernel function
//   |
// VMRUN again
//   |-- hardware sets GIF=1
//   |-- pending interrupts delivered
//   |-- guest resumes

The CPU automatically clears GIF on #VMEXIT and sets it on VMRUN. I never have to touch it. The entire exit handler runs in this interrupt-free window. No preemption, no context switches, no surprise NMIs.

Why this is a problem

If nothing can interrupt me, that means nothing can wake me up either. If my exit handler blocks -- waits on a spinlock, calls a function that internally acquires a lock, or does anything that might need an interrupt to make progress -- the system hangs. Not a BSOD. A hard lock. No NMI watchdog because NMIs are masked. No SMI-based recovery because SMIs are masked. The only fix is the reset button.

This rules out a huge list of operations I might want to do in an exit handler:

// things I CANNOT do at GIF=0

ExAllocatePool()      // may acquire pool lock
ExFreePool()           // same lock
MmProbeAndLockPages() // may page fault
KeAcquireSpinLock()    // cross-core deadlock risk
ZwReadFile()           // needs interrupt completion
DbgPrint()             // serial port may need interrupt

// basically: no kernel API is safe at GIF=0
// the only safe operations are:
//   - reading/writing my own pre-allocated memory
//   - reading/writing MSRs
//   - reading/writing guest register state in the VMCB
//   - reading physical memory I've already mapped

Defensive RDMSR

One of the first things that bit me was RDMSR faulting on unknown MSR addresses. When the guest reads an MSR that doesn't exist, the hardware would normally #GP. But at GIF=0, a #GP in the host is fatal -- there's no IDT handler that can safely run. The CPU double-faults, then triple-faults, and the machine resets.

My solution: return zero for any MSR I don't explicitly know about.

fn safe_rdmsr(msr: u32) -> u64 {
    match msr {
        // known MSRs -- read real hardware
        0xC0000080 => native_rdmsr(0xC0000080),
        0xC0000081..=0xC0000084 => native_rdmsr(msr),
        0x00000174..=0x00000176 => native_rdmsr(msr),
        0x000001A0 => native_rdmsr(msr), // IA32_MISC_ENABLE
        0x00000277 => native_rdmsr(msr), // PAT

        // unknown MSR -- return 0, don't fault
        _ => {
            0
        }
    }
}
Returning zero for unknown MSRs is technically incorrect -- some MSRs have meaningful default values. But at GIF=0, correctness takes a back seat to survival. A wrong MSR value causes a guest bug. A #GP at GIF=0 causes a machine reset. I'll take the bug.

No-alloc NPF handling

Nested page faults are the most complex exit to handle. Sometimes I need to split a 2MB page into 512 individual 4KB pages to hide a single page in the middle. Normally that means allocating a page table page -- 4KB from the pool. But I can't allocate at GIF=0.

I pre-allocate a pool of page table pages at driver init time. My NPF handler uses try_map_4k_noalloc which pulls from this pre-allocated pool. If the pool is empty, I can't split the page -- I fall back to mapping the entire 2MB page and accepting the leak. This has never actually happened in practice because I pre-allocate 256 pages and typical usage is under 50.

fn try_map_4k_noalloc(
    npt: &mut NptRoot,
    gpa: PhysAddr,
    hpa: Option<PhysAddr>,
) -> Result<(), ()> {
    // check if we need to split a 2MB page
    let pde = npt.get_pde(gpa);
    if pde.is_large_page() {
        // need a PT page from the pre-allocated pool
        let pt_page = match PT_PAGE_POOL.try_alloc() {
            Some(p) => p,
            None => return Err(()),  // pool exhausted
        };
        split_large_page(pde, pt_page);
    }

    // now set the individual 4KB PTE
    let pte = npt.get_pte(gpa);
    match hpa {
        Some(addr) => pte.set_mapping(addr),
        None => pte.clear(),
    }
    Ok(())
}

Why not STGI?

I could call STGI (Set Global Interrupt Flag) at the top of my exit handler. That would re-enable interrupts and let me use kernel APIs normally. Some hypervisors do this. I don't, for two reasons.

First, if an NMI arrives while I'm modifying NPT tables, the NMI handler runs in the host context with my half-modified page tables. That's a corruption vector. I'd need to save and restore NPT state around every operation that could be interrupted, which is complex and error-prone.

Second, if an SMI arrives, the CPU enters SMM. If I'm running SmmInfect, my SMM code sees the host context in a weird state -- exit handler half-done, guest registers half-restored. The interaction between my hypervisor's exit handler and my SMM code would be a nightmare to reason about.

Staying at GIF=0 is simpler. Everything is atomic from the perspective of the rest of the system. My exit handler runs to completion without any possibility of interruption. The constraints are harsh -- no allocation, no blocking, no faulting -- but the simplicity is worth it.

NoirVisor confirmation

I spent a while wondering if I was doing something unusual by running at GIF=0 the whole time. Then I looked at NoirVisor's CVM (Customizable Virtual Machine) implementation. Same pattern. Exit handler runs at GIF=0. No STGI before dispatch. Pre-allocated resources. Defensive MSR reads.

This isn't an accident. It's the natural consequence of the constraint: GIF=0 is the only way to guarantee atomicity of the exit handler. Once you accept that constraint, everything else follows. Pre-allocate what you need. Don't call the kernel. Return zero for unknowns. Keep the handler fast.

Host TSC compensation/IBPB/vmcb_clean are skipped for guest exits. The guest VMCB doesn't need host stealth measures. Those only matter for the blue-pill VMCB that the Windows kernel is running under. This keeps the guest exit path minimal and fast -- important when you can't afford to spend extra time at GIF=0.

The swap storm lesson

I learned the hard way how painful GIF=0 debugging is. During stage 9 development, I had a "swap storm" where the host kept switching to the guest VMCB after every single host exit. The guest would yield on VMEXIT_INTR, the host would process one exit and immediately re-arm the guest, the guest would yield again. Infinite cycle, system frozen.

The fix was a cooldown counter: after a guest yield, set host_exit_countdown = 50. The host processes 50 exits before considering re-arming the guest. This gives the host time to handle pending work without getting stuck in a ping-pong loop. Simple, but I only discovered it after staring at a frozen machine for three hours.

← prev next →