Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

mm, page_alloc: use check_pages_enabled static key to check tail pages

Commit 700d2e9a36b9 ("mm, page_alloc: reduce page alloc/free sanity
checks") has introduced a new static key check_pages_enabled to control
when struct pages are sanity checked during allocation and freeing. Mel
Gorman suggested that free_tail_pages_check() could use this static key as
well, instead of relying on CONFIG_DEBUG_VM. That makes sense, so do
that. Also rename the function to free_tail_page_prepare() because it
works on a single tail page and has a struct page preparation component as
well as the optional checking component.

Also remove some unnecessary unlikely() within static_branch_unlikely()
statements that Mel pointed out for commit 700d2e9a36b9.

Link: https://lkml.kernel.org/r/20230405142840.11068-1-vbabka@suse.cz
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Suggested-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Cc: Alexander Halbuer <halbuer@sra.uni-hannover.de>
Cc: Kees Cook <keescook@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Vlastimil Babka and committed by
Andrew Morton
8666925c 27da93d8

+6 -6
+1 -1
mm/hugetlb_vmemmap.c
··· 264 264 * How many struct page structs need to be reset. When we reuse the head 265 265 * struct page, the special metadata (e.g. page->flags or page->mapping) 266 266 * cannot copy to the tail struct page structs. The invalid value will be 267 - * checked in the free_tail_pages_check(). In order to avoid the message 267 + * checked in the free_tail_page_prepare(). In order to avoid the message 268 268 * of "corrupted mapping in tail page". We need to reset at least 3 (one 269 269 * head struct page struct and two tail struct page structs) struct page 270 270 * structs.
+5 -5
mm/page_alloc.c
··· 1131 1131 return true; 1132 1132 } 1133 1133 1134 - static int free_tail_pages_check(struct page *head_page, struct page *page) 1134 + static int free_tail_page_prepare(struct page *head_page, struct page *page) 1135 1135 { 1136 1136 struct folio *folio = (struct folio *)head_page; 1137 1137 int ret = 1; ··· 1142 1142 */ 1143 1143 BUILD_BUG_ON((unsigned long)LIST_POISON1 & 1); 1144 1144 1145 - if (!IS_ENABLED(CONFIG_DEBUG_VM)) { 1145 + if (!static_branch_unlikely(&check_pages_enabled)) { 1146 1146 ret = 0; 1147 1147 goto out; 1148 1148 } ··· 1276 1276 ClearPageHasHWPoisoned(page); 1277 1277 for (i = 1; i < (1 << order); i++) { 1278 1278 if (compound) 1279 - bad += free_tail_pages_check(page, page + i); 1279 + bad += free_tail_page_prepare(page, page + i); 1280 1280 if (is_check_pages_enabled()) { 1281 - if (unlikely(free_page_is_bad(page + i))) { 1281 + if (free_page_is_bad(page + i)) { 1282 1282 bad++; 1283 1283 continue; 1284 1284 } ··· 1627 1627 for (int i = 0; i < (1 << order); i++) { 1628 1628 struct page *p = page + i; 1629 1629 1630 - if (unlikely(check_new_page(p))) 1630 + if (check_new_page(p)) 1631 1631 return true; 1632 1632 } 1633 1633 }