Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

bpf, arm: Optimize ALU ARSH K using asr immediate instruction

This patch adds an optimization that uses the asr immediate instruction
for BPF_ALU BPF_ARSH BPF_K, rather than loading the immediate to
a temporary register. This is similar to existing code for handling
BPF_ALU BPF_{LSH,RSH} BPF_K. This optimization saves two instructions
and is more consistent with LSH and RSH.

Example of the code generated for BPF_ALU32_IMM(BPF_ARSH, BPF_REG_0, 5)
before the optimization:

2c: mov r8, #5
30: mov r9, #0
34: asr r0, r0, r8

and after optimization:

2c: asr r0, r0, #5

Tested on QEMU using lib/test_bpf and test_verifier.

Co-developed-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200501020210.32294-3-luke.r.nels@gmail.com

authored by

Luke Nelson and committed by
Daniel Borkmann
c648c9c7 cf48db69

+10 -3
+7 -3
arch/arm/net/bpf_jit_32.c
··· 795 795 case BPF_RSH: 796 796 emit(ARM_LSR_I(rd, rd, val), ctx); 797 797 break; 798 + case BPF_ARSH: 799 + emit(ARM_ASR_I(rd, rd, val), ctx); 800 + break; 798 801 case BPF_NEG: 799 802 emit(ARM_RSB_I(rd, rd, val), ctx); 800 803 break; ··· 1411 1408 case BPF_ALU | BPF_MUL | BPF_X: 1412 1409 case BPF_ALU | BPF_LSH | BPF_X: 1413 1410 case BPF_ALU | BPF_RSH | BPF_X: 1414 - case BPF_ALU | BPF_ARSH | BPF_K: 1415 1411 case BPF_ALU | BPF_ARSH | BPF_X: 1416 1412 case BPF_ALU64 | BPF_ADD | BPF_K: 1417 1413 case BPF_ALU64 | BPF_ADD | BPF_X: ··· 1467 1465 case BPF_ALU64 | BPF_MOD | BPF_K: 1468 1466 case BPF_ALU64 | BPF_MOD | BPF_X: 1469 1467 goto notyet; 1470 - /* dst = dst >> imm */ 1471 1468 /* dst = dst << imm */ 1472 - case BPF_ALU | BPF_RSH | BPF_K: 1469 + /* dst = dst >> imm */ 1470 + /* dst = dst >> imm (signed) */ 1473 1471 case BPF_ALU | BPF_LSH | BPF_K: 1472 + case BPF_ALU | BPF_RSH | BPF_K: 1473 + case BPF_ALU | BPF_ARSH | BPF_K: 1474 1474 if (unlikely(imm > 31)) 1475 1475 return -EINVAL; 1476 1476 if (imm)
+3
arch/arm/net/bpf_jit_32.h
··· 94 94 #define ARM_INST_LSR_I 0x01a00020 95 95 #define ARM_INST_LSR_R 0x01a00030 96 96 97 + #define ARM_INST_ASR_I 0x01a00040 98 + #define ARM_INST_ASR_R 0x01a00050 99 + 97 100 #define ARM_INST_MOV_R 0x01a00000 98 101 #define ARM_INST_MOVS_R 0x01b00000 99 102 #define ARM_INST_MOV_I 0x03a00000