Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

arm64: entry: Make the kpti trampoline's kpti sequence optional

Spectre-BHB needs to add sequences to the vectors. Having one global
set of vectors is a problem for big/little systems where the sequence
is costly on cpus that are not vulnerable.

Making the vectors per-cpu in the style of KVM's bh_harden_hyp_vecs
requires the vectors to be generated by macros.

Make the kpti re-mapping of the kernel optional, so the macros can be
used without kpti.

Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>

+12 -6
+12 -6
arch/arm64/kernel/entry.S
··· 646 646 sub \dst, \dst, PAGE_SIZE 647 647 .endm 648 648 649 - .macro tramp_ventry, vector_start, regsize 649 + .macro tramp_ventry, vector_start, regsize, kpti 650 650 .align 7 651 651 1: 652 + .if \kpti == 1 652 653 .if \regsize == 64 653 654 msr tpidrro_el0, x30 // Restored in kernel_ventry 654 655 .endif ··· 672 671 alternative_if_not ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM 673 672 prfm plil1strm, [x30, #(1b - \vector_start)] 674 673 alternative_else_nop_endif 674 + 675 675 msr vbar_el1, x30 676 - add x30, x30, #(1b - \vector_start + 4) 677 676 isb 677 + .else 678 + ldr x30, =vectors 679 + .endif // \kpti == 1 680 + 681 + add x30, x30, #(1b - \vector_start + 4) 678 682 ret 679 683 .org 1b + 128 // Did we overflow the ventry slot? 680 684 .endm ··· 697 691 sb 698 692 .endm 699 693 700 - .macro generate_tramp_vector 694 + .macro generate_tramp_vector, kpti 701 695 .Lvector_start\@: 702 696 .space 0x400 703 697 704 698 .rept 4 705 - tramp_ventry .Lvector_start\@, 64 699 + tramp_ventry .Lvector_start\@, 64, \kpti 706 700 .endr 707 701 .rept 4 708 - tramp_ventry .Lvector_start\@, 32 702 + tramp_ventry .Lvector_start\@, 32, \kpti 709 703 .endr 710 704 .endm 711 705 ··· 716 710 .pushsection ".entry.tramp.text", "ax" 717 711 .align 11 718 712 SYM_CODE_START_NOALIGN(tramp_vectors) 719 - generate_tramp_vector 713 + generate_tramp_vector kpti=1 720 714 SYM_CODE_END(tramp_vectors) 721 715 722 716 SYM_CODE_START(tramp_exit_native)