path: root/arch/x86/mm/pgtable.c
diff options
authorDave Hansen <>2013-04-12 16:23:54 -0700
committerLinus Torvalds <>2013-04-12 16:56:47 -0700
commit1de14c3c5cbc9bb17e9dcc648cda51c0c85d54b9 (patch)
treeab6010ad78195651c39a8ca3829ba1b95451762f /arch/x86/mm/pgtable.c
parentbf81710c4b6e2df2cc047f7c8e1f342511904b74 (diff)
x86-32: Fix possible incomplete TLB invalidate with PAE pagetables
This patch attempts to fix: The symptom is a crash and messages like this: chrome: Corrupted page table at address 34a03000 *pdpt = 0000000000000000 *pde = 0000000000000000 Bad pagetable: 000f [#1] PREEMPT SMP Ingo guesses this got introduced by commit 611ae8e3f520 ("x86/tlb: enable tlb flush range support for x86") since that code started to free unused pagetables. On x86-32 PAE kernels, that new code has the potential to free an entire PMD page and will clear one of the four page-directory-pointer-table (aka pgd_t entries). The hardware aggressively "caches" these top-level entries and invlpg does not actually affect the CPU's copy. If we clear one we *HAVE* to do a full TLB flush, otherwise we might continue using a freed pmd page. (note, we do this properly on the population side in pud_populate()). This patch tracks whenever we clear one of these entries in the 'struct mmu_gather', and ensures that we follow up with a full tlb flush. BTW, I disassembled and checked that: if (tlb->fullmm == 0) and if (!tlb->fullmm && !tlb->need_flush_all) generate essentially the same code, so there should be zero impact there to the !PAE case. Signed-off-by: Dave Hansen <> Cc: Peter Anvin <> Cc: Ingo Molnar <> Cc: Artem S Tashkinov <> Signed-off-by: Linus Torvalds <>
Diffstat (limited to 'arch/x86/mm/pgtable.c')
1 files changed, 7 insertions, 0 deletions
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 193350b51f9..17fda6a8b3c 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -58,6 +58,13 @@ void ___pte_free_tlb(struct mmu_gather *tlb, struct page *pte)
void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd)
paravirt_release_pmd(__pa(pmd) >> PAGE_SHIFT);
+ /*
+ * NOTE! For PAE, any changes to the top page-directory-pointer-table
+ * entries need a full cr3 reload to flush.
+ */
+#ifdef CONFIG_X86_PAE
+ tlb->need_flush_all = 1;
tlb_remove_page(tlb, virt_to_page(pmd));