Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

locking/atomic/x86: Introduce the read64_nonatomic macro to x86_32 with cx8

As described in commit:

e73c4e34a0e9 ("locking/atomic/x86: Introduce arch_atomic64_read_nonatomic() to x86_32")

the value preload before the CMPXCHG loop does not need to be atomic.

Introduce the read64_nonatomic assembly macro to load the value from a
atomic_t location in a faster non-atomic way and use it in
atomic64_cx8_32.S.

Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20240605181424.3228-1-ubizjak@gmail.com

authored by

Uros Bizjak and committed by
Ingo Molnar
6e30a7c9 51835949

+7 -2
+7 -2
arch/x86/lib/atomic64_cx8_32.S
··· 16 16 cmpxchg8b (\reg) 17 17 .endm 18 18 19 + .macro read64_nonatomic reg 20 + movl (\reg), %eax 21 + movl 4(\reg), %edx 22 + .endm 23 + 19 24 SYM_FUNC_START(atomic64_read_cx8) 20 25 read64 %ecx 21 26 RET ··· 56 51 movl %edx, %edi 57 52 movl %ecx, %ebp 58 53 59 - read64 %ecx 54 + read64_nonatomic %ecx 60 55 1: 61 56 movl %eax, %ebx 62 57 movl %edx, %ecx ··· 84 79 SYM_FUNC_START(atomic64_\func\()_return_cx8) 85 80 pushl %ebx 86 81 87 - read64 %esi 82 + read64_nonatomic %esi 88 83 1: 89 84 movl %eax, %ebx 90 85 movl %edx, %ecx