Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

bpf: Move bpf_dispatcher function out of ftrace locations

The dispatcher function is attached/detached to trampoline by
dispatcher update function. At the same time it's available as
ftrace attachable function.

After discussion [1] the proposed solution is to use compiler
attributes to alter bpf_dispatcher_##name##_func function:

- remove it from being instrumented with __no_instrument_function__
attribute, so ftrace has no track of it

- but still generate 5 nop instructions with patchable_function_entry(5)
attribute, which are expected by bpf_arch_text_poke used by
dispatcher update function

Enabling HAVE_DYNAMIC_FTRACE_NO_PATCHABLE option for x86, so
__patchable_function_entries functions are not part of ftrace/mcount
locations.

Adding attributes to bpf_dispatcher_XXX function on x86_64 so it's
kept out of ftrace locations and has 5 byte nop generated at entry.

These attributes need to be arch specific as pointed out by Ilya
Leoshkevic in here [2].

The dispatcher image is generated only for x86_64 arch, so the
code can stay as is for other archs.

[1] https://lore.kernel.org/bpf/20220722110811.124515-1-jolsa@kernel.org/
[2] https://lore.kernel.org/bpf/969a14281a7791c334d476825863ee449964dd0c.camel@linux.ibm.com/

Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/bpf/20220903131154.420467-3-jolsa@kernel.org

authored by

Jiri Olsa and committed by
Daniel Borkmann
ceea991a 9440155c

+8
+1
arch/x86/Kconfig
··· 284 284 select PROC_PID_ARCH_STATUS if PROC_FS 285 285 select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX 286 286 imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI 287 + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE 287 288 288 289 config INSTRUCTION_DECODER 289 290 def_bool y
+7
include/linux/bpf.h
··· 977 977 }, \ 978 978 } 979 979 980 + #ifdef CONFIG_X86_64 981 + #define BPF_DISPATCHER_ATTRIBUTES __attribute__((patchable_function_entry(5))) 982 + #else 983 + #define BPF_DISPATCHER_ATTRIBUTES 984 + #endif 985 + 980 986 #define DEFINE_BPF_DISPATCHER(name) \ 987 + notrace BPF_DISPATCHER_ATTRIBUTES \ 981 988 noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ 982 989 const void *ctx, \ 983 990 const struct bpf_insn *insnsi, \