Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

bpf, arm64: use separate register for state in stxr

Will reported that in BPF_XADD we must use a different register in stxr
instruction for the status flag due to otherwise CONSTRAINED UNPREDICTABLE
behavior per architecture. Reference manual says [1]:

If s == t, then one of the following behaviors must occur:

* The instruction is UNDEFINED.
* The instruction executes as a NOP.
* The instruction performs the store to the specified address, but
the value stored is UNKNOWN.

Thus, use a different temporary register for the status flag to fix it.

Disassembly extract from test 226/STX_XADD_DW from test_bpf.ko:

[...]
0000003c: c85f7d4b ldxr x11, [x10]
00000040: 8b07016b add x11, x11, x7
00000044: c80c7d4b stxr w12, x11, [x10]
00000048: 35ffffac cbnz w12, 0x0000003c
[...]

[1] https://static.docs.arm.com/ddi0487/b/DDI0487B_a_armv8_arm.pdf, p.6132

Fixes: 85f68fe89832 ("bpf, arm64: implement jiting of BPF_XADD")
Reported-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

authored by

Daniel Borkmann and committed by
David S. Miller
7005cade e173db36

+5 -2
+5 -2
arch/arm64/net/bpf_jit_comp.c
··· 36 36 #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) 37 37 #define TMP_REG_2 (MAX_BPF_JIT_REG + 1) 38 38 #define TCALL_CNT (MAX_BPF_JIT_REG + 2) 39 + #define TMP_REG_3 (MAX_BPF_JIT_REG + 3) 39 40 40 41 /* Map BPF registers to A64 registers */ 41 42 static const int bpf2a64[] = { ··· 58 57 /* temporary registers for internal BPF JIT */ 59 58 [TMP_REG_1] = A64_R(10), 60 59 [TMP_REG_2] = A64_R(11), 60 + [TMP_REG_3] = A64_R(12), 61 61 /* tail_call_cnt */ 62 62 [TCALL_CNT] = A64_R(26), 63 63 /* temporary register for blinding constants */ ··· 321 319 const u8 src = bpf2a64[insn->src_reg]; 322 320 const u8 tmp = bpf2a64[TMP_REG_1]; 323 321 const u8 tmp2 = bpf2a64[TMP_REG_2]; 322 + const u8 tmp3 = bpf2a64[TMP_REG_3]; 324 323 const s16 off = insn->off; 325 324 const s32 imm = insn->imm; 326 325 const int i = insn - ctx->prog->insnsi; ··· 692 689 emit(A64_PRFM(tmp, PST, L1, STRM), ctx); 693 690 emit(A64_LDXR(isdw, tmp2, tmp), ctx); 694 691 emit(A64_ADD(isdw, tmp2, tmp2, src), ctx); 695 - emit(A64_STXR(isdw, tmp2, tmp, tmp2), ctx); 692 + emit(A64_STXR(isdw, tmp2, tmp, tmp3), ctx); 696 693 jmp_offset = -3; 697 694 check_imm19(jmp_offset); 698 - emit(A64_CBNZ(0, tmp2, jmp_offset), ctx); 695 + emit(A64_CBNZ(0, tmp3, jmp_offset), ctx); 699 696 break; 700 697 701 698 /* R0 = ntohx(*(size *)(((struct sk_buff *)R6)->data + imm)) */