Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

arm64: entry: always restore x0 from the stack on syscall return

We have a micro-optimisation on the fast syscall return path where we
take care to keep x0 live with the return value from the syscall so that
we can avoid restoring it from the stack. The benefit of doing this is
fairly suspect, since we will be restoring x1 from the stack anyway
(which lives adjacent in the pt_regs structure) and the only additional
cost is saving x0 back to pt_regs after the syscall handler, which could
be seen as a poor man's prefetch.

More importantly, this causes issues with the context tracking code.

The ct_user_enter macro ends up branching into C code, which is free to
use x0 as a scratch register and consequently leads to us returning junk
back to userspace as the syscall return value. Rather than special case
the context-tracking code, this patch removes the questionable
optimisation entirely.

Cc: <stable@vger.kernel.org>
Cc: Larry Bassel <larry.bassel@linaro.org>
Cc: Kevin Hilman <khilman@linaro.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>

+6 -11
+6 -11
arch/arm64/kernel/entry.S
··· 116 116 */ 117 117 .endm 118 118 119 - .macro kernel_exit, el, ret = 0 119 + .macro kernel_exit, el 120 120 ldp x21, x22, [sp, #S_PC] // load ELR, SPSR 121 121 .if \el == 0 122 122 ct_user_enter ··· 143 143 .endif 144 144 msr elr_el1, x21 // set up the return data 145 145 msr spsr_el1, x22 146 - .if \ret 147 - ldr x1, [sp, #S_X1] // preserve x0 (syscall return) 148 - .else 149 146 ldp x0, x1, [sp, #16 * 0] 150 - .endif 151 147 ldp x2, x3, [sp, #16 * 1] 152 148 ldp x4, x5, [sp, #16 * 2] 153 149 ldp x6, x7, [sp, #16 * 3] ··· 606 610 */ 607 611 ret_fast_syscall: 608 612 disable_irq // disable interrupts 613 + str x0, [sp, #S_X0] // returned x0 609 614 ldr x1, [tsk, #TI_FLAGS] // re-check for syscall tracing 610 615 and x2, x1, #_TIF_SYSCALL_WORK 611 616 cbnz x2, ret_fast_syscall_trace 612 617 and x2, x1, #_TIF_WORK_MASK 613 - cbnz x2, fast_work_pending 618 + cbnz x2, work_pending 614 619 enable_step_tsk x1, x2 615 - kernel_exit 0, ret = 1 620 + kernel_exit 0 616 621 ret_fast_syscall_trace: 617 622 enable_irq // enable interrupts 618 - b __sys_trace_return 623 + b __sys_trace_return_skipped // we already saved x0 619 624 620 625 /* 621 626 * Ok, we need to do extra processing, enter the slow path. 622 627 */ 623 - fast_work_pending: 624 - str x0, [sp, #S_X0] // returned x0 625 628 work_pending: 626 629 tbnz x1, #TIF_NEED_RESCHED, work_resched 627 630 /* TIF_SIGPENDING, TIF_NOTIFY_RESUME or TIF_FOREIGN_FPSTATE case */ ··· 644 649 cbnz x2, work_pending 645 650 enable_step_tsk x1, x2 646 651 no_work_pending: 647 - kernel_exit 0, ret = 0 652 + kernel_exit 0 648 653 ENDPROC(ret_to_user) 649 654 650 655 /*