Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

arm64: Turn the MMU-on sequence into a macro

Turning the MMU on is a popular sport in the arm64 kernel, and
we do it more than once, or even twice. As we are about to add
even more, let's turn it into a macro.

No expected functional change.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: David Brazdil <dbrazdil@google.com>
Link: https://lore.kernel.org/r/20210208095732.3267263-4-maz@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>

authored by

Marc Zyngier and committed by
Will Deacon
8cc8a324 b161f924

+22 -26
+17
arch/arm64/include/asm/assembler.h
··· 676 676 .endm 677 677 678 678 /* 679 + * Set SCTLR_EL1 to the passed value, and invalidate the local icache 680 + * in the process. This is called when setting the MMU on. 681 + */ 682 + .macro set_sctlr_el1, reg 683 + msr sctlr_el1, \reg 684 + isb 685 + /* 686 + * Invalidate the local I-cache so that any instructions fetched 687 + * speculatively from the PoC are discarded, since they may have 688 + * been dynamically patched at the PoU. 689 + */ 690 + ic iallu 691 + dsb nsh 692 + isb 693 + .endm 694 + 695 + /* 679 696 * Check whether to yield to another runnable task from kernel mode NEON code 680 697 * (which runs with preemption disabled). 681 698 *
+4 -15
arch/arm64/kernel/head.S
··· 703 703 offset_ttbr1 x1, x3 704 704 msr ttbr1_el1, x1 // load TTBR1 705 705 isb 706 - msr sctlr_el1, x0 707 - isb 708 - /* 709 - * Invalidate the local I-cache so that any instructions fetched 710 - * speculatively from the PoC are discarded, since they may have 711 - * been dynamically patched at the PoU. 712 - */ 713 - ic iallu 714 - dsb nsh 715 - isb 706 + 707 + set_sctlr_el1 x0 708 + 716 709 ret 717 710 SYM_FUNC_END(__enable_mmu) 718 711 ··· 876 883 tlbi vmalle1 // Remove any stale TLB entries 877 884 dsb nsh 878 885 879 - msr sctlr_el1, x19 // re-enable the MMU 880 - isb 881 - ic iallu // flush instructions fetched 882 - dsb nsh // via old mapping 883 - isb 886 + set_sctlr_el1 x19 // re-enable the MMU 884 887 885 888 bl __relocate_kernel 886 889 #endif
+1 -11
arch/arm64/mm/proc.S
··· 291 291 /* We're done: fire up the MMU again */ 292 292 mrs x17, sctlr_el1 293 293 orr x17, x17, #SCTLR_ELx_M 294 - msr sctlr_el1, x17 295 - isb 296 - 297 - /* 298 - * Invalidate the local I-cache so that any instructions fetched 299 - * speculatively from the PoC are discarded, since they may have 300 - * been dynamically patched at the PoU. 301 - */ 302 - ic iallu 303 - dsb nsh 304 - isb 294 + set_sctlr_el1 x17 305 295 306 296 /* Set the flag to zero to indicate that we're all done */ 307 297 str wzr, [flag_ptr]