Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

function_graph: Move graph notrace bit to shadow stack global var

The use of the task->trace_recursion for the logic used for the function
graph no-trace was a bit of an abuse of that variable. Now that there
exists global vars that are per stack for registered graph traces, use
that instead.

Link: https://lore.kernel.org/linux-trace-kernel/171509107907.162236.6564679266777519065.stgit@devnote2
Link: https://lore.kernel.org/linux-trace-kernel/20240603190823.796709456@goodmis.org

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Florent Revest <revest@chromium.org>
Cc: Martin KaFai Lau <martin.lau@linux.dev>
Cc: bpf <bpf@vger.kernel.org>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Guo Ren <guoren@kernel.org>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>

+15 -11
-7
include/linux/trace_recursion.h
··· 44 44 */ 45 45 TRACE_IRQ_BIT, 46 46 47 - /* 48 - * To implement set_graph_notrace, if this bit is set, we ignore 49 - * function graph tracing of called functions, until the return 50 - * function is called to clear it. 51 - */ 52 - TRACE_GRAPH_NOTRACE_BIT, 53 - 54 47 /* Used to prevent recursion recording from recursing. */ 55 48 TRACE_RECORD_RECURSION_BIT, 56 49 };
+9
kernel/trace/trace.h
··· 919 919 920 920 TRACE_GRAPH_DEPTH_START_BIT, 921 921 TRACE_GRAPH_DEPTH_END_BIT, 922 + 923 + /* 924 + * To implement set_graph_notrace, if this bit is set, we ignore 925 + * function graph tracing of called functions, until the return 926 + * function is called to clear it. 927 + */ 928 + TRACE_GRAPH_NOTRACE_BIT, 922 929 }; 930 + 931 + #define TRACE_GRAPH_NOTRACE (1 << TRACE_GRAPH_NOTRACE_BIT) 923 932 924 933 static inline unsigned long ftrace_graph_depth(unsigned long *task_var) 925 934 {
+6 -4
kernel/trace/trace_functions_graph.c
··· 130 130 int trace_graph_entry(struct ftrace_graph_ent *trace, 131 131 struct fgraph_ops *gops) 132 132 { 133 + unsigned long *task_var = fgraph_get_task_var(gops); 133 134 struct trace_array *tr = gops->private; 134 135 struct trace_array_cpu *data; 135 136 unsigned long flags; ··· 139 138 int ret; 140 139 int cpu; 141 140 142 - if (trace_recursion_test(TRACE_GRAPH_NOTRACE_BIT)) 141 + if (*task_var & TRACE_GRAPH_NOTRACE) 143 142 return 0; 144 143 145 144 /* ··· 150 149 * returning from the function. 151 150 */ 152 151 if (ftrace_graph_notrace_addr(trace->func)) { 153 - trace_recursion_set(TRACE_GRAPH_NOTRACE_BIT); 152 + *task_var |= TRACE_GRAPH_NOTRACE_BIT; 154 153 /* 155 154 * Need to return 1 to have the return called 156 155 * that will clear the NOTRACE bit. ··· 241 240 void trace_graph_return(struct ftrace_graph_ret *trace, 242 241 struct fgraph_ops *gops) 243 242 { 243 + unsigned long *task_var = fgraph_get_task_var(gops); 244 244 struct trace_array *tr = gops->private; 245 245 struct trace_array_cpu *data; 246 246 unsigned long flags; ··· 251 249 252 250 ftrace_graph_addr_finish(gops, trace); 253 251 254 - if (trace_recursion_test(TRACE_GRAPH_NOTRACE_BIT)) { 255 - trace_recursion_clear(TRACE_GRAPH_NOTRACE_BIT); 252 + if (*task_var & TRACE_GRAPH_NOTRACE) { 253 + *task_var &= ~TRACE_GRAPH_NOTRACE; 256 254 return; 257 255 } 258 256