(I'm so sorry!) Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

bpf, arm64: Place trampoline arguments by the arm64 calling convention

calc_arg_aux(), save_args() and restore_args() number argument registers and stack slots consecutively, while AAPCS64 rounds both up to an even one for an argument aligned to 16 bytes: an __int128, or an aggregate holding one. The argument that takes the hole, and every argument after it, is saved from and restored to the wrong place, so a bpf program attached to such a function reads a neighbouring eightbyte, and the arguments handed on to the original function are shifted. Take the position of each argument slot from bpf_jit_place_args() and keep it in struct arg_aux. save_args() then reads a slot from the register, or the incoming stack slot, and restore_args() puts it back there; the outgoing area built for the original function mirrors the incoming one, hole and all. A bpf program still cannot read an __int128 argument itself: it is an integer wider than eight bytes, which btf_ctx_access() refuses, so the program is rejected whatever the trampoline does with the argument. What a program can read, and what the placement puts right, is an aggregate holding an __int128, allowed as a struct, and any argument that follows a 16-byte aligned one. Both were taken from the wrong place before. Bound the argument slots rather than the arguments while here. btf_distill_func_proto() only limits the count although a by-value argument may take two slots, so, similar to x86-64, support up to MAX_BPF_FUNC_ARGS argument slots and refuse a function with more. Signed-off-by: Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/r/20260912195252.990578-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>


Author Yonghong Song Committer Alexei Starovoitov Date Commit Parent a3195057
+54 -24
1 changed file
+54 -24
1 changed file