Merge branch 'bpf-support-by-value-struct-and-__int128-arguments'
Yonghong Song says: ==================== bpf: Support by-value struct and __int128 arguments A global function or a kfunc taking a struct or union by value is rejected today: Arg#1 type STRUCT in tar() is not supported yet. Unrecognized R2 type STRUCT and an __int128 argument is accepted but mis-counted: the compiler passes it in two registers while the verifier gives it one, so every argument after it is checked against the wrong register and the program is rejected for a register its source never names. The compiler passes such a value in one argument register per eightbyte, so a value of at most 16 bytes arrives in one or two consecutive registers. This series teaches the verifier to describe that: sub->args[] gains one entry per argument slot rather than one per BTF parameter, and the kfunc argument walk counts slots the same way. A value may only be composed of scalars, as a pointer would reach the callee as an opaque scalar, losing the provenance and reference tracking that make it safe to use. A global function still has no stack arguments, so all of its slots have to fit in the argument registers. For a kfunc the two calling conventions have to agree, and they do not always. Where the BPF convention splits a value between the last argument register and the stack, x86_64 moves the whole of it to the stack and gives the register to the argument that follows, while arm64 rounds the register number up to an even one for a 16-byte aligned value. For stack arguments, both x86_64 and arm64 will have 16-byte alignment for those 16-byte aligned parameters. Otherwise the alignment will be 8-byte. This is implemented in x86_64 and arm64 jit. arm64 has some issues with trampolines due to certain 16-byte align requirement, similar to 16-byte align for kfunc arguments. calc_arg_aux(), save_args() and restore_args() all need changes to cope with 16-byte alignment. Patch 1 fixed a pre-existing bug in bpf_kfunc_stack_access_bytes(). Patch 2 records the __int128 failure as it stands today, which patch 6 turns into a success. Patches 3 to 6 rename some fields, index the arguments of a global function by slot and accept a by-value struct and an __int128. Patches 8 and 9 have the same for a kfunc call. Patches 10 to 12 place the arguments per the kernel calling convention in the x86-64 and arm64 JITs. Patches 13 to 15 are the tests. Tested on x86-64 and arm64 with test_progs. The by-value struct tests are built with clang, as GCC passes an aggregate by invisible reference, and some of them need __BPF_FEATURE_STACK_ARGUMENT. Changelog: v3 -> v4: - v3: https://lore.kernel.org/bpf/20260911154914.2004336-1-yonghong.song@linux.dev/ - Rebase on top master branch. Mostly no change except the commit "Recognize by-value struct and __int128 kfunc arguments" which needs some adjustment. - Commit "bpf, arm64: Place trampoline arguments by the arm64 calling convention" is new. The commit intends to fix the bug when any argument has 16-byte alignment requirement. v2 -> v3: - v2: https://lore.kernel.org/bpf/20260909062522.4001896-1-yonghong.song@linux.dev/ - Rename test name from verifier_int_arg to verifier_aggregate_arg. - Rename from arg_cnt to arg_slot_cnt to make codes easy to understand. - Fix an issue for by-value struct arguments up to 16 bytes where the bound check, e.g. number of slots is more than 5 or not, etc, is missing after all arguments are processed. - Increase local slot size from MAX_BPF_FUNC_ARGS to 2 * MAX_BPF_FUNC_ARGS to avoid out-of-bound write. - One codegen improvement for arm64 jit. - Added a not-to-merge patch (patch 15) to workaround some tests. The pahole patch https://lore.kernel.org/bpf/20260911040955.339939-1-yonghong.song@linux.dev/ is pending. v1 -> v2: - v1: https://lore.kernel.org/bpf/20260904050957.3976119-1-yonghong.song@linux.dev/ - Fix a few issues due to potential out of bound access. - Fix a few missed cases for kfunc with '> 8 byte' arguments. - Add jit support for x86_64 and arm64. Previously certain functions are rejected, and now these functions can succeed. ==================== Link: https://patch.msgid.link/20260912195156.980886-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>