2026-02-18 // 13 min

NPT Deep Dive: Identity Mapping, Page Splitting, and the Hidden Page Table

nptmemoryamd-svm

Nested Page Tables are the foundation of everything the hypervisor does. Without NPT, I can't hide my own memory from the guest. I can't redirect code execution. I can't intercept specific physical pages. Every other feature -- CPUID masking, MSR filtering, TSC compensation -- those are all important, but NPT is the one that makes the hypervisor invisible.

AMD's NPT adds a second level of address translation. The guest has its own page tables (CR3-based) that translate virtual to "guest physical" addresses. NPT then translates those guest physical addresses to real host physical addresses. The guest never sees the NPT -- it's entirely managed by the hypervisor through the VMCB's npt_cr3 field.

I spent a full week getting NPT right. The page table structure is the same 4-level hierarchy as regular x86-64 paging, but the implications of getting anything wrong are immediate and catastrophic -- a bad NPT entry means a triple fault and instant machine reset.

The 4-level hierarchy

NPT uses the same PML4 -> PDPT -> PD -> PT structure as regular paging. Each level is a 4KB page containing 512 entries. Each entry is 8 bytes. The walk starts from the physical address in npt_cr3 and uses 9-bit slices of the guest physical address to index each level.

// NPT page table walk for guest physical address → host physical
// same 4-level structure as regular x86-64 paging
fn npt_walk(npt_cr3: u64, gpa: u64) -> u64 {
    let pml4_idx = (gpa >> 39) & 0x1FF;
    let pdpt_idx = (gpa >> 30) & 0x1FF;
    let pd_idx   = (gpa >> 21) & 0x1FF;
    let pt_idx   = (gpa >> 12) & 0x1FF;

    let pml4e = read_phys(npt_cr3 + pml4_idx * 8);
    let pdpte = read_phys((pml4e & 0xFFFFF000) + pdpt_idx * 8);
    let pde   = read_phys((pdpte & 0xFFFFF000) + pd_idx * 8);

    // check PS (Page Size) bit -- if set, this is a 2MB large page
    if pde & (1 << 7) != 0 {
        return (pde & 0xFFFE00000) | (gpa & 0x1FFFFF);
    }
    let pte = read_phys((pde & 0xFFFFF000) + pt_idx * 8);
    (pte & 0xFFFFF000) | (gpa & 0xFFF)
}

A full 4KB page walk touches 4 physical pages (one per level). A 2MB large page walk only touches 3 -- the PD entry has the PS (Page Size) bit set, so the walk stops there and the remaining 21 bits of the GPA index directly into the 2MB region. That's why I use 2MB pages wherever possible -- fewer levels to walk means fewer memory accesses means less TLB pressure.

Identity mapping all of RAM

The first thing I do is build NPT as a pure identity map: guest physical address X maps to host physical address X for all of RAM. This means the guest sees memory exactly as it would on bare metal. Then I selectively modify entries to hide my own pages or redirect specific addresses.

For efficiency, I map everything with 2MB large pages. On my system with 32GB of RAM, that's 16384 entries across 32 PD pages, plus a handful of PDPT and PML4 entries. The entire NPT structure fits in about 200KB of memory.

Every NPT entry has permission bits -- read, write, execute (NX). I set all entries to RWX by default (identity map). When I want to hide a page, I clear all permission bits -- the entry is still present but any access triggers a Nested Page Fault (#NPF). When I want to intercept writes to a page, I clear the write bit. When I want to redirect execution, I set up dual NPTs -- one for reads, one for execution.

Splitting 2MB pages to 4KB

Here's the problem: I map all of RAM with 2MB large pages for efficiency. But when I need to hide a single 4KB page (like one of my VMCB pages that happens to fall within a 2MB region), I can't just unmap the whole 2MB -- that would destroy whatever OS data shares that region.

So I split the 2MB large page into 512 individual 4KB page table entries. 511 of them identity-map to the same physical addresses they covered before. The one entry I care about gets modified -- unmapped, redirected, or permission-changed.

// Split a 2MB large page into 512 x 4KB pages
// so I can modify permissions on a single 4KB page
fn split_2mb_to_4kb(
    pde: &mut NptEntry,
    pages: &mut Vec<PhysAddr>,
) {
    let base_pa = pde.pfn() << 21; // 2MB-aligned base
    let pt_page = alloc_npt_page();
    pages.push(pt_page);

    let pt = pt_page as *mut [NptEntry; 512];
    for i in 0..512 {
        // identity map each 4KB sub-page
        (*pt)[i] = NptEntry::new(
            base_pa + i * 0x1000,
            NptFlags::PRESENT | NptFlags::RWX,
        );
    }
    // replace 2MB PDE with pointer to new PT
    // clear PS bit -- this is now a 4KB page table pointer
    *pde = NptEntry::new(
        pt_page,
        NptFlags::PRESENT | NptFlags::RWX, // no PS bit
    );
}

After the split, the 2MB region behaves identically from the guest's perspective -- every GPA still maps to the same HPA. But now I can modify individual 4KB entries within that region. I use this to unmap my own hypervisor pages, to set up NPT hooks on specific code pages, and to intercept MMIO regions like HPET.

The pages[] array and npt_pa_to_va()

Every page I allocate for NPT structures goes into a tracking array. This is critical for cleanup -- when the hypervisor devirtualizes, I need to free every NPT page or I leak kernel pool memory. But it also solves a more immediate problem.

When I walk the NPT to find a PDE or PTE, find_pde() gives me a physical address (because page table entries contain physical addresses). But I need a virtual address to actually read/write the entry from my handler code. The kernel function MmGetVirtualForPhysical only works for addresses that the memory manager knows about -- it returns null for pool-allocated pages that aren't in the PFN database's virtual address mappings.

So I wrote npt_pa_to_va() that searches the pages[] array to find which allocation contains a given physical address and computes the corresponding virtual address. It's a linear scan -- not fast, but NPT modifications are rare (only during setup or hook installation), so it doesn't matter.

The NPT is the only data structure in the hypervisor where I work with physical addresses directly. Everything else -- VMCBs, MSR permission bitmaps, host stacks -- I allocate and access via virtual addresses. NPT entries are special because the hardware walks them using physical addresses, so every entry stores a physical page frame number. This is why the PA-to-VA translation is necessary.

Hiding the hypervisor

Once I have fine-grained 4KB control over the NPT, hiding the hypervisor is straightforward. For every page that contains hypervisor data -- VMCBs, host save areas, MSR permission bitmaps, NPT pages themselves, per-CPU stacks, the driver image -- I find the corresponding NPT PTE and clear all permission bits.

When the guest tries to access one of these pages, the CPU triggers a #NPF (Nested Page Fault, exit code 0x400). My handler checks the faulting GPA, sees it's one of my hidden pages, and injects a zero-filled response. The guest sees a page of zeros. It can scan every byte of physical memory and find nothing.

The number of hidden pages is small -- maybe 200-300 pages total for all hypervisor structures across all 16 cores. That's about 1.2MB of memory that simply doesn't exist from the guest's perspective. The rest of the 32GB is identity-mapped and behaves exactly like bare metal.

Performance cost

NPT adds a second page table walk for every TLB miss. On bare metal, a TLB miss walks 4 levels (PML4 -> PDPT -> PD -> PT). With NPT active, each of those 4 guest page table accesses itself requires an NPT walk. That's up to 24 memory accesses for a single TLB miss instead of 4. AMD's documentation calls this the "2D page walk" problem.

In practice, it's mitigated by the TLB. AMD's TLB caches NPT translations alongside guest translations, and the ASID (Address Space ID) system means the hypervisor can avoid TLB flushes on VMRUN/#VMEXIT. On my Ryzen 7, the NPT overhead is measurable in microbenchmarks but invisible in real workloads. Games still hit the same frame rates. Windows boot time is unchanged.

The 2MB large pages help enormously here. A 2MB NPT entry covers the same range as 512 4KB entries but only adds 3 levels to the walk instead of 4. That's fewer memory accesses, fewer TLB entries consumed, and better coverage per entry. I only split to 4KB when I absolutely need per-page granularity.

10 / DMA ← 12 / CPUID Masking →