V_INTR_MASKING: The Bit That Prevents CLOCK_WATCHDOG_TIMEOUT

One bit. Bit 24 of the VMCB's v_intr field at offset 0x060. If I don't set it when running a guest VM, the system hits CLOCK_WATCHDOG_TIMEOUT within seconds. No warning, no graceful degradation -- just an instant bugcheck 0x101.

I spent hours debugging what I thought was a timing issue. The guest would boot, start executing real code, and then the entire machine would BSOD. Every single time. The bugcheck pointed at a clock interrupt that never fired on one of the cores, which made me think I had a TSC problem or an APIC routing bug. I was wrong.

What V_INTR_MASKING actually does

In AMD SVM, V_INTR_MASKING controls which RFLAGS.IF the CPU uses for physical interrupt delivery during guest execution. Without it, the guest's RFLAGS.IF controls whether physical interrupts cause a #VMEXIT. With it, the host's RFLAGS.IF at VMRUN time controls that decision instead.

// VMCB control area, offset 0x060
// v_intr field layout:
//   bits [7:0]   = V_TPR
//   bit  [8]     = V_IRQ
//   bits [19:16] = V_INTR_VECTOR
//   bit  [24]    = V_INTR_MASKING  <-- THIS ONE

vmcb.control.v_intr |= 1u64 << 24;

The distinction matters because guest operating systems toggle IF constantly. Spinlock acquisition, interrupt handlers, critical sections -- the guest clears IF all the time. Without V_INTR_MASKING, every time the guest runs with IF=0, physical interrupts don't cause VMEXITs. The clock interrupt gets silently suppressed on that core.

Why it causes CLOCK_WATCHDOG_TIMEOUT

Windows expects every logical processor to acknowledge clock interrupts within a deadline. The HAL tracks this. If a core goes too long without servicing its clock tick, the kernel raises bugcheck 0x101 -- CLOCK_WATCHDOG_TIMEOUT. The parameter block tells me exactly which core stopped responding.

// What Windows sees without V_INTR_MASKING:
//
// 1. Guest runs on core 14
// 2. Guest enters spinlock_acquire()
// 3. Guest clears RFLAGS.IF  (cli)
// 4. Physical clock interrupt fires
// 5. Guest IF=0, so NO #VMEXIT
//    ^^^ interrupt is suppressed
// 6. Host clock ISR never fires on core 14
// 7. Windows watchdog times out
// 8. CLOCK_WATCHDOG_TIMEOUT

// What happens WITH V_INTR_MASKING:
//
// 1. Guest runs on core 14
// 2. Guest clears RFLAGS.IF
// 3. Physical clock interrupt fires
// 4. Host IF=1 (set before VMRUN), so #VMEXIT fires
// 5. Host clock ISR runs normally
// 6. Watchdog happy

Setting IF before VMRUN

Once V_INTR_MASKING is set, I need the host's RFLAGS.IF to be 1 when VMRUN executes. But I can't just use STI -- there's an STI shadow that delays interrupt delivery by one instruction, which can cause subtle races. Instead I modify RFLAGS directly through the stack.

// In the vmrun assembly stub:
pushfq
or qword ptr [rsp], 0x200
popfq
// now VMRUN executes with host IF=1
vmrun rax

This is what KVM does too. The PUSHFQ/OR/POPFQ sequence sets IF without the one-instruction shadow window that STI creates. After #VMEXIT, hardware restores the host RFLAGS with IF=1, STGI re-enables interrupts, and any pending interrupt fires immediately.

The real debugging lesson

The bug persisted for hours because of a duplicate VMCB setup path. I had guest_setup.rs correctly setting bit 24 in v_intr. But then lib.rs line 556 was overwriting v_intr with a fresh value -- without bit 24 -- as part of a later initialization step. My correct fix was getting silently undone.

I found it by logging every exit. When I finally looked at the exit trace, there were zero VMEXIT_INTR (0x60) exits. On a running system, clock interrupts fire hundreds of times per second per core. Zero interrupt exits means interrupts are being suppressed, not that they aren't happening. That's when it clicked -- the masking was wrong, not the timing.

The duplicate setup path

The most frustrating part of this bug was that I had the correct fix from the beginning. guest_setup.rs had the line vmcb.control.v_intr |= 1u64 << 24 right where it should be. I'd read the manual, understood the requirement, written the code. It should have worked.

But lib.rs had a second VMCB initialization block that ran after guest_setup.rs. This block was copy-pasted from an earlier stage of the project when I was only doing blue-pill virtualization. It re-initialized v_intr to set up virtual interrupt delivery for the host -- without including bit 24, because the blue-pill host doesn't need V_INTR_MASKING (the host's IF behavior doesn't matter when you are the host).

So the correct value was being written, then silently overwritten. I had to diff the VMCB state at two different points in initialization to catch it. Once I saw that v_intr was 0x01000000 after guest_setup.rs and 0x00000000 after lib.rs, it was obvious.

// The overwrite chain:
//
// guest_setup.rs:42   vmcb.v_intr |= 1 << 24   ← correct
//   v_intr = 0x0100_0000
//
// lib.rs:556          vmcb.v_intr = 0x00         ← overwrites!
//   v_intr = 0x0000_0000
//
// VMRUN executes with v_intr = 0
// V_INTR_MASKING not set → guest IF controls phys interrupts
// → CLOCK_WATCHDOG_TIMEOUT

One bit

The fix is literally one line. vmcb.control.v_intr |= 1u64 << 24. But finding which line to write, and finding the duplicate that was erasing it, took an entire afternoon. The VMCB has hundreds of configuration bits. Each one can cause a different flavor of system instability. And the symptoms rarely point at the actual cause -- a missing masking bit looks exactly like a timing problem until I count the interrupt exits.

This is now in my mental checklist for any new guest VM setup. Step one: set V_INTR_MASKING. Step two: verify it's still set after all the initialization code has run. Step three: if the BSOD comes back, check for duplicate initialization paths. The VMCB should be configured in exactly one place. Any second write to the same field is a bug waiting to happen.

I've since refactored the VMCB setup so that guest configuration lives entirely in guest_setup.rs and lib.rs never touches guest VMCB fields after the initial hand-off. Single source of truth. No more silent overwrites.