Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Merge tag 'trace-v5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:
"The majority of the changes are for fixes and clean ups.

Notable changes:

- Rework trace event triggers code to be easier to interact with.

- Support for embedding bootconfig with the kernel (as suppose to
having it embedded in initram). This is useful for embedded boards
without initram disks.

- Speed up boot by parallelizing the creation of tracefs files.

- Allow absolute ring buffer timestamps handle timestamps that use
more than 59 bits.

- Added new tracing clock "TAI" (International Atomic Time)

- Have weak functions show up in available_filter_function list as:
__ftrace_invalid_address___<invalid-offset> instead of using the
name of the function before it"

* tag 'trace-v5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (52 commits)
ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to avoid adding weak function
tracing: Fix comments for event_trigger_separate_filter()
x86/traceponit: Fix comment about irq vector tracepoints
x86,tracing: Remove unused headers
ftrace: Clean up hash direct_functions on register failures
tracing: Fix comments of create_filter()
tracing: Disable kcov on trace_preemptirq.c
tracing: Initialize integer variable to prevent garbage return value
ftrace: Fix typo in comment
ftrace: Remove return value of ftrace_arch_modify_*()
tracing: Cleanup code by removing init "char *name"
tracing: Change "char *" string form to "char []"
tracing/timerlat: Do not wakeup the thread if the trace stops at the IRQ
tracing/timerlat: Print stacktrace in the IRQ handler if needed
tracing/timerlat: Notify IRQ new max latency only if stop tracing is set
kprobes: Fix build errors with CONFIG_KRETPROBES=n
tracing: Fix return value of trace_pid_write()
tracing: Fix potential double free in create_var_ref()
tracing: Use strim() to remove whitespace instead of doing it manually
ftrace: Deal with error return code of the ftrace_process_locs() function
...

+827 -651
+28 -3
Documentation/admin-guide/bootconfig.rst
··· 158 158 Boot Kernel With a Boot Config 159 159 ============================== 160 160 161 - Since the boot configuration file is loaded with initrd, it will be added 162 - to the end of the initrd (initramfs) image file with padding, size, 163 - checksum and 12-byte magic word as below. 161 + There are two options to boot the kernel with bootconfig: attaching the 162 + bootconfig to the initrd image or embedding it in the kernel itself. 163 + 164 + Attaching a Boot Config to Initrd 165 + --------------------------------- 166 + 167 + Since the boot configuration file is loaded with initrd by default, 168 + it will be added to the end of the initrd (initramfs) image file with 169 + padding, size, checksum and 12-byte magic word as below. 164 170 165 171 [initrd][bootconfig][padding][size(le32)][checksum(le32)][#BOOTCONFIG\n] 166 172 ··· 202 196 Then add "bootconfig" on the normal kernel command line to tell the 203 197 kernel to look for the bootconfig at the end of the initrd file. 204 198 199 + Embedding a Boot Config into Kernel 200 + ----------------------------------- 201 + 202 + If you can not use initrd, you can also embed the bootconfig file in the 203 + kernel by Kconfig options. In this case, you need to recompile the kernel 204 + with the following configs:: 205 + 206 + CONFIG_BOOT_CONFIG_EMBED=y 207 + CONFIG_BOOT_CONFIG_EMBED_FILE="/PATH/TO/BOOTCONFIG/FILE" 208 + 209 + ``CONFIG_BOOT_CONFIG_EMBED_FILE`` requires an absolute path or a relative 210 + path to the bootconfig file from source tree or object tree. 211 + The kernel will embed it as the default bootconfig. 212 + 213 + Just as when attaching the bootconfig to the initrd, you need ``bootconfig`` 214 + option on the kernel command line to enable the embedded bootconfig. 215 + 216 + Note that even if you set this option, you can override the embedded 217 + bootconfig by another bootconfig which attached to the initrd. 205 218 206 219 Kernel parameters via Boot Config 207 220 =================================
+12
Documentation/trace/ftrace.rst
··· 517 517 processing should be able to handle them. See comments in the 518 518 ktime_get_boot_fast_ns() function for more information. 519 519 520 + tai: 521 + This is the tai clock (CLOCK_TAI) and is derived from the wall- 522 + clock time. However, this clock does not experience 523 + discontinuities and backwards jumps caused by NTP inserting leap 524 + seconds. Since the clock access is designed for use in tracing, 525 + side effects are possible. The clock access may yield wrong 526 + readouts in case the internal TAI offset is updated e.g., caused 527 + by setting the system time or using adjtimex() with an offset. 528 + These effects are rare and post processing should be able to 529 + handle them. See comments in the ktime_get_tai_fast_ns() 530 + function for more information. 531 + 520 532 To set a clock, simply echo the clock name into this file:: 521 533 522 534 # echo global > trace_clock
+3 -2
Documentation/trace/timerlat-tracer.rst
··· 74 74 - stop_tracing_total_us: stop the system tracing if a 75 75 timer latency at the *thread* context is higher than the configured 76 76 value happens. Writing 0 disables this option. 77 - - print_stack: save the stack of the IRQ occurrence, and print 78 - it after the *thread context* event". 77 + - print_stack: save the stack of the IRQ occurrence. The stack is printed 78 + after the *thread context* event, or at the IRQ handler if *stop_tracing_us* 79 + is hit. 79 80 80 81 timerlat and osnoise 81 82 ----------------------------
+3 -2
MAINTAINERS
··· 7517 7517 F: Documentation/admin-guide/bootconfig.rst 7518 7518 F: fs/proc/bootconfig.c 7519 7519 F: include/linux/bootconfig.h 7520 + F: lib/bootconfig-data.S 7520 7521 F: lib/bootconfig.c 7521 7522 F: tools/bootconfig/* 7522 7523 F: tools/bootconfig/scripts/* ··· 20120 20119 S: Maintained 20121 20120 T: git git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git 20122 20121 F: Documentation/trace/ftrace.rst 20123 - F: arch/*/*/*/ftrace.h 20124 - F: arch/*/kernel/ftrace.c 20122 + F: arch/*/*/*/*ftrace* 20123 + F: arch/*/*/*ftrace* 20125 20124 F: fs/tracefs/ 20126 20125 F: include/*/ftrace.h 20127 20126 F: include/linux/trace*.h
+2 -4
arch/arm/kernel/ftrace.c
··· 79 79 return (unsigned long)&ftrace_regs_caller_from_init; 80 80 } 81 81 82 - int ftrace_arch_code_modify_prepare(void) 82 + void ftrace_arch_code_modify_prepare(void) 83 83 { 84 - return 0; 85 84 } 86 85 87 - int ftrace_arch_code_modify_post_process(void) 86 + void ftrace_arch_code_modify_post_process(void) 88 87 { 89 88 /* Make sure any TLB misses during machine stop are cleared. */ 90 89 flush_tlb_all(); 91 - return 0; 92 90 } 93 91 94 92 static unsigned long ftrace_call_replace(unsigned long pc, unsigned long addr,
+2 -4
arch/riscv/kernel/ftrace.c
··· 12 12 #include <asm/patch.h> 13 13 14 14 #ifdef CONFIG_DYNAMIC_FTRACE 15 - int ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) 15 + void ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) 16 16 { 17 17 mutex_lock(&text_mutex); 18 - return 0; 19 18 } 20 19 21 - int ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) 20 + void ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) 22 21 { 23 22 mutex_unlock(&text_mutex); 24 - return 0; 25 23 } 26 24 27 25 static int ftrace_check_current_call(unsigned long hook_pos,
+1 -2
arch/s390/kernel/ftrace.c
··· 225 225 ftrace_modify_all_code(command); 226 226 } 227 227 228 - int ftrace_arch_code_modify_post_process(void) 228 + void ftrace_arch_code_modify_post_process(void) 229 229 { 230 230 /* 231 231 * Flush any pre-fetched instructions on all 232 232 * CPUs to make the new code visible. 233 233 */ 234 234 text_poke_sync_lock(); 235 - return 0; 236 235 } 237 236 238 237 #ifdef CONFIG_MODULES
+7
arch/x86/include/asm/ftrace.h
··· 9 9 # define MCOUNT_ADDR ((unsigned long)(__fentry__)) 10 10 #define MCOUNT_INSN_SIZE 5 /* sizeof mcount call */ 11 11 12 + /* Ignore unused weak functions which will have non zero offsets */ 13 + #ifdef CONFIG_HAVE_FENTRY 14 + # include <asm/ibt.h> 15 + /* Add offset for endbr64 if IBT enabled */ 16 + # define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE 17 + #endif 18 + 12 19 #ifdef CONFIG_DYNAMIC_FTRACE 13 20 #define ARCH_SUPPORTS_FTRACE_OPS 1 14 21 #endif
+2 -4
arch/x86/kernel/ftrace.c
··· 37 37 38 38 static int ftrace_poke_late = 0; 39 39 40 - int ftrace_arch_code_modify_prepare(void) 40 + void ftrace_arch_code_modify_prepare(void) 41 41 __acquires(&text_mutex) 42 42 { 43 43 /* ··· 47 47 */ 48 48 mutex_lock(&text_mutex); 49 49 ftrace_poke_late = 1; 50 - return 0; 51 50 } 52 51 53 - int ftrace_arch_code_modify_post_process(void) 52 + void ftrace_arch_code_modify_post_process(void) 54 53 __releases(&text_mutex) 55 54 { 56 55 /* ··· 60 61 text_poke_finish(); 61 62 ftrace_poke_late = 0; 62 63 mutex_unlock(&text_mutex); 63 - return 0; 64 64 } 65 65 66 66 static const char *ftrace_nop_replace(void)
-6
arch/x86/kernel/tracepoint.c
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 /* 3 - * Code for supporting irq vector tracepoints. 4 - * 5 3 * Copyright (C) 2013 Seiji Aguchi <seiji.aguchi@hds.com> 6 - * 7 4 */ 8 5 #include <linux/jump_label.h> 9 6 #include <linux/atomic.h> 10 7 11 - #include <asm/hw_irq.h> 12 - #include <asm/desc.h> 13 8 #include <asm/trace/exceptions.h> 14 - #include <asm/trace/irq_vectors.h> 15 9 16 10 DEFINE_STATIC_KEY_FALSE(trace_pagefault_key); 17 11
+10
include/linux/bootconfig.h
··· 289 289 /* XBC cleanup data structures */ 290 290 void __init xbc_exit(void); 291 291 292 + /* XBC embedded bootconfig data in kernel */ 293 + #ifdef CONFIG_BOOT_CONFIG_EMBED 294 + const char * __init xbc_get_embedded_bootconfig(size_t *size); 295 + #else 296 + static inline const char *xbc_get_embedded_bootconfig(size_t *size) 297 + { 298 + return NULL; 299 + } 300 + #endif 301 + 292 302 #endif
+2 -2
include/linux/ftrace.h
··· 452 452 453 453 #ifdef CONFIG_DYNAMIC_FTRACE 454 454 455 - int ftrace_arch_code_modify_prepare(void); 456 - int ftrace_arch_code_modify_post_process(void); 455 + void ftrace_arch_code_modify_prepare(void); 456 + void ftrace_arch_code_modify_post_process(void); 457 457 458 458 enum ftrace_bug_type { 459 459 FTRACE_BUG_UNKNOWN,
+1 -1
include/linux/kprobes.h
··· 424 424 int register_kretprobes(struct kretprobe **rps, int num); 425 425 void unregister_kretprobes(struct kretprobe **rps, int num); 426 426 427 - #ifdef CONFIG_KRETPROBE_ON_RETHOOK 427 + #if defined(CONFIG_KRETPROBE_ON_RETHOOK) || !defined(CONFIG_KRETPROBES) 428 428 #define kprobe_flush_task(tk) do {} while (0) 429 429 #else 430 430 void kprobe_flush_task(struct task_struct *tk);
+20 -1
init/Kconfig
··· 1338 1338 1339 1339 config BOOT_CONFIG 1340 1340 bool "Boot config support" 1341 - select BLK_DEV_INITRD 1341 + select BLK_DEV_INITRD if !BOOT_CONFIG_EMBED 1342 1342 help 1343 1343 Extra boot config allows system admin to pass a config file as 1344 1344 complemental extension of kernel cmdline when booting. ··· 1347 1347 See <file:Documentation/admin-guide/bootconfig.rst> for details. 1348 1348 1349 1349 If unsure, say Y. 1350 + 1351 + config BOOT_CONFIG_EMBED 1352 + bool "Embed bootconfig file in the kernel" 1353 + depends on BOOT_CONFIG 1354 + help 1355 + Embed a bootconfig file given by BOOT_CONFIG_EMBED_FILE in the 1356 + kernel. Usually, the bootconfig file is loaded with the initrd 1357 + image. But if the system doesn't support initrd, this option will 1358 + help you by embedding a bootconfig file while building the kernel. 1359 + 1360 + If unsure, say N. 1361 + 1362 + config BOOT_CONFIG_EMBED_FILE 1363 + string "Embedded bootconfig file path" 1364 + depends on BOOT_CONFIG_EMBED 1365 + help 1366 + Specify a bootconfig file which will be embedded to the kernel. 1367 + This bootconfig will be used if there is no initrd or no other 1368 + bootconfig in the initrd. 1350 1369 1351 1370 config INITRAMFS_PRESERVE_MTIME 1352 1371 bool "Preserve cpio archive mtimes in initramfs"
+19 -19
init/main.c
··· 266 266 early_param("loglevel", loglevel); 267 267 268 268 #ifdef CONFIG_BLK_DEV_INITRD 269 - static void * __init get_boot_config_from_initrd(u32 *_size, u32 *_csum) 269 + static void * __init get_boot_config_from_initrd(size_t *_size) 270 270 { 271 271 u32 size, csum; 272 272 char *data; ··· 300 300 return NULL; 301 301 } 302 302 303 + if (xbc_calc_checksum(data, size) != csum) { 304 + pr_err("bootconfig checksum failed\n"); 305 + return NULL; 306 + } 307 + 303 308 /* Remove bootconfig from initramfs/initrd */ 304 309 initrd_end = (unsigned long)data; 305 310 if (_size) 306 311 *_size = size; 307 - if (_csum) 308 - *_csum = csum; 309 312 310 313 return data; 311 314 } 312 315 #else 313 - static void * __init get_boot_config_from_initrd(u32 *_size, u32 *_csum) 316 + static void * __init get_boot_config_from_initrd(size_t *_size) 314 317 { 315 318 return NULL; 316 319 } ··· 410 407 static void __init setup_boot_config(void) 411 408 { 412 409 static char tmp_cmdline[COMMAND_LINE_SIZE] __initdata; 413 - const char *msg; 414 - int pos; 415 - u32 size, csum; 416 - char *data, *err; 417 - int ret; 410 + const char *msg, *data; 411 + int pos, ret; 412 + size_t size; 413 + char *err; 418 414 419 415 /* Cut out the bootconfig data even if we have no bootconfig option */ 420 - data = get_boot_config_from_initrd(&size, &csum); 416 + data = get_boot_config_from_initrd(&size); 417 + /* If there is no bootconfig in initrd, try embedded one. */ 418 + if (!data) 419 + data = xbc_get_embedded_bootconfig(&size); 421 420 422 421 strlcpy(tmp_cmdline, boot_command_line, COMMAND_LINE_SIZE); 423 422 err = parse_args("bootconfig", tmp_cmdline, NULL, 0, 0, 0, NULL, ··· 438 433 } 439 434 440 435 if (size >= XBC_DATA_MAX) { 441 - pr_err("bootconfig size %d greater than max size %d\n", 442 - size, XBC_DATA_MAX); 443 - return; 444 - } 445 - 446 - if (xbc_calc_checksum(data, size) != csum) { 447 - pr_err("bootconfig checksum failed\n"); 436 + pr_err("bootconfig size %ld greater than max size %d\n", 437 + (long)size, XBC_DATA_MAX); 448 438 return; 449 439 } 450 440 ··· 452 452 msg, pos); 453 453 } else { 454 454 xbc_get_info(&ret, NULL); 455 - pr_info("Load bootconfig: %d bytes %d nodes\n", size, ret); 455 + pr_info("Load bootconfig: %ld bytes %d nodes\n", (long)size, ret); 456 456 /* keys starting with "kernel." are passed via cmdline */ 457 457 extra_command_line = xbc_make_cmdline("kernel"); 458 458 /* Also, "init." keys are init arguments */ ··· 471 471 static void __init setup_boot_config(void) 472 472 { 473 473 /* Remove bootconfig data from initrd */ 474 - get_boot_config_from_initrd(NULL, NULL); 474 + get_boot_config_from_initrd(NULL); 475 475 } 476 476 477 477 static int __init warn_bootconfig(char *str)
+71 -73
kernel/kprobes.c
··· 1257 1257 preempt_enable(); 1258 1258 } 1259 1259 1260 - #if !defined(CONFIG_KRETPROBE_ON_RETHOOK) 1261 - static void free_rp_inst_rcu(struct rcu_head *head) 1262 - { 1263 - struct kretprobe_instance *ri = container_of(head, struct kretprobe_instance, rcu); 1264 - 1265 - if (refcount_dec_and_test(&ri->rph->ref)) 1266 - kfree(ri->rph); 1267 - kfree(ri); 1268 - } 1269 - NOKPROBE_SYMBOL(free_rp_inst_rcu); 1270 - 1271 - static void recycle_rp_inst(struct kretprobe_instance *ri) 1272 - { 1273 - struct kretprobe *rp = get_kretprobe(ri); 1274 - 1275 - if (likely(rp)) 1276 - freelist_add(&ri->freelist, &rp->freelist); 1277 - else 1278 - call_rcu(&ri->rcu, free_rp_inst_rcu); 1279 - } 1280 - NOKPROBE_SYMBOL(recycle_rp_inst); 1281 - 1282 - /* 1283 - * This function is called from delayed_put_task_struct() when a task is 1284 - * dead and cleaned up to recycle any kretprobe instances associated with 1285 - * this task. These left over instances represent probed functions that 1286 - * have been called but will never return. 1287 - */ 1288 - void kprobe_flush_task(struct task_struct *tk) 1289 - { 1290 - struct kretprobe_instance *ri; 1291 - struct llist_node *node; 1292 - 1293 - /* Early boot, not yet initialized. */ 1294 - if (unlikely(!kprobes_initialized)) 1295 - return; 1296 - 1297 - kprobe_busy_begin(); 1298 - 1299 - node = __llist_del_all(&tk->kretprobe_instances); 1300 - while (node) { 1301 - ri = container_of(node, struct kretprobe_instance, llist); 1302 - node = node->next; 1303 - 1304 - recycle_rp_inst(ri); 1305 - } 1306 - 1307 - kprobe_busy_end(); 1308 - } 1309 - NOKPROBE_SYMBOL(kprobe_flush_task); 1310 - 1311 - static inline void free_rp_inst(struct kretprobe *rp) 1312 - { 1313 - struct kretprobe_instance *ri; 1314 - struct freelist_node *node; 1315 - int count = 0; 1316 - 1317 - node = rp->freelist.head; 1318 - while (node) { 1319 - ri = container_of(node, struct kretprobe_instance, freelist); 1320 - node = node->next; 1321 - 1322 - kfree(ri); 1323 - count++; 1324 - } 1325 - 1326 - if (refcount_sub_and_test(count, &rp->rph->ref)) { 1327 - kfree(rp->rph); 1328 - rp->rph = NULL; 1329 - } 1330 - } 1331 - #endif /* !CONFIG_KRETPROBE_ON_RETHOOK */ 1332 - 1333 1260 /* Add the new probe to 'ap->list'. */ 1334 1261 static int add_new_kprobe(struct kprobe *ap, struct kprobe *p) 1335 1262 { ··· 1855 1928 #ifdef CONFIG_KRETPROBES 1856 1929 1857 1930 #if !defined(CONFIG_KRETPROBE_ON_RETHOOK) 1931 + static void free_rp_inst_rcu(struct rcu_head *head) 1932 + { 1933 + struct kretprobe_instance *ri = container_of(head, struct kretprobe_instance, rcu); 1934 + 1935 + if (refcount_dec_and_test(&ri->rph->ref)) 1936 + kfree(ri->rph); 1937 + kfree(ri); 1938 + } 1939 + NOKPROBE_SYMBOL(free_rp_inst_rcu); 1940 + 1941 + static void recycle_rp_inst(struct kretprobe_instance *ri) 1942 + { 1943 + struct kretprobe *rp = get_kretprobe(ri); 1944 + 1945 + if (likely(rp)) 1946 + freelist_add(&ri->freelist, &rp->freelist); 1947 + else 1948 + call_rcu(&ri->rcu, free_rp_inst_rcu); 1949 + } 1950 + NOKPROBE_SYMBOL(recycle_rp_inst); 1951 + 1952 + /* 1953 + * This function is called from delayed_put_task_struct() when a task is 1954 + * dead and cleaned up to recycle any kretprobe instances associated with 1955 + * this task. These left over instances represent probed functions that 1956 + * have been called but will never return. 1957 + */ 1958 + void kprobe_flush_task(struct task_struct *tk) 1959 + { 1960 + struct kretprobe_instance *ri; 1961 + struct llist_node *node; 1962 + 1963 + /* Early boot, not yet initialized. */ 1964 + if (unlikely(!kprobes_initialized)) 1965 + return; 1966 + 1967 + kprobe_busy_begin(); 1968 + 1969 + node = __llist_del_all(&tk->kretprobe_instances); 1970 + while (node) { 1971 + ri = container_of(node, struct kretprobe_instance, llist); 1972 + node = node->next; 1973 + 1974 + recycle_rp_inst(ri); 1975 + } 1976 + 1977 + kprobe_busy_end(); 1978 + } 1979 + NOKPROBE_SYMBOL(kprobe_flush_task); 1980 + 1981 + static inline void free_rp_inst(struct kretprobe *rp) 1982 + { 1983 + struct kretprobe_instance *ri; 1984 + struct freelist_node *node; 1985 + int count = 0; 1986 + 1987 + node = rp->freelist.head; 1988 + while (node) { 1989 + ri = container_of(node, struct kretprobe_instance, freelist); 1990 + node = node->next; 1991 + 1992 + kfree(ri); 1993 + count++; 1994 + } 1995 + 1996 + if (refcount_sub_and_test(count, &rp->rph->ref)) { 1997 + kfree(rp->rph); 1998 + rp->rph = NULL; 1999 + } 2000 + } 2001 + 1858 2002 /* This assumes the 'tsk' is the current task or the is not running. */ 1859 2003 static kprobe_opcode_t *__kretprobe_find_ret_addr(struct task_struct *tsk, 1860 2004 struct llist_node **cur)
+4
kernel/trace/Makefile
··· 31 31 GCOV_PROFILE := y 32 32 endif 33 33 34 + # Functions in this file could be invoked from early interrupt 35 + # code and produce random code coverage. 36 + KCOV_INSTRUMENT_trace_preemptirq.o := n 37 + 34 38 CFLAGS_bpf_trace.o := -I$(src) 35 39 36 40 CFLAGS_trace_benchmark.o := -I$(src)
+174 -36
kernel/trace/ftrace.c
··· 45 45 #include "trace_output.h" 46 46 #include "trace_stat.h" 47 47 48 + #define FTRACE_INVALID_FUNCTION "__ftrace_invalid_address__" 49 + 48 50 #define FTRACE_WARN_ON(cond) \ 49 51 ({ \ 50 52 int ___r = cond; \ ··· 121 119 ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub; 122 120 struct ftrace_ops global_ops; 123 121 124 - /* Defined by vmlinux.lds.h see the commment above arch_ftrace_ops_list_func for details */ 122 + /* Defined by vmlinux.lds.h see the comment above arch_ftrace_ops_list_func for details */ 125 123 void ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip, 126 124 struct ftrace_ops *op, struct ftrace_regs *fregs); 127 125 ··· 954 952 static __init void ftrace_profile_tracefs(struct dentry *d_tracer) 955 953 { 956 954 struct ftrace_profile_stat *stat; 957 - struct dentry *entry; 958 955 char *name; 959 956 int ret; 960 957 int cpu; ··· 984 983 } 985 984 } 986 985 987 - entry = tracefs_create_file("function_profile_enabled", 988 - TRACE_MODE_WRITE, d_tracer, NULL, 989 - &ftrace_profile_fops); 990 - if (!entry) 991 - pr_warn("Could not create tracefs 'function_profile_enabled' entry\n"); 986 + trace_create_file("function_profile_enabled", 987 + TRACE_MODE_WRITE, d_tracer, NULL, 988 + &ftrace_profile_fops); 992 989 } 993 990 994 991 #else /* CONFIG_FUNCTION_PROFILER */ ··· 2706 2707 * archs can override this function if they must do something 2707 2708 * before the modifying code is performed. 2708 2709 */ 2709 - int __weak ftrace_arch_code_modify_prepare(void) 2710 + void __weak ftrace_arch_code_modify_prepare(void) 2710 2711 { 2711 - return 0; 2712 2712 } 2713 2713 2714 2714 /* 2715 2715 * archs can override this function if they must do something 2716 2716 * after the modifying code is performed. 2717 2717 */ 2718 - int __weak ftrace_arch_code_modify_post_process(void) 2718 + void __weak ftrace_arch_code_modify_post_process(void) 2719 2719 { 2720 - return 0; 2721 2720 } 2722 2721 2723 2722 void ftrace_modify_all_code(int command) ··· 2801 2804 2802 2805 static void ftrace_run_update_code(int command) 2803 2806 { 2804 - int ret; 2805 - 2806 - ret = ftrace_arch_code_modify_prepare(); 2807 - FTRACE_WARN_ON(ret); 2808 - if (ret) 2809 - return; 2807 + ftrace_arch_code_modify_prepare(); 2810 2808 2811 2809 /* 2812 2810 * By default we use stop_machine() to modify the code. ··· 2811 2819 */ 2812 2820 arch_ftrace_update_code(command); 2813 2821 2814 - ret = ftrace_arch_code_modify_post_process(); 2815 - FTRACE_WARN_ON(ret); 2822 + ftrace_arch_code_modify_post_process(); 2816 2823 } 2817 2824 2818 2825 static void ftrace_run_modify_code(struct ftrace_ops *ops, int command, ··· 3622 3631 seq_printf(m, " ->%pS", ptr); 3623 3632 } 3624 3633 3634 + #ifdef FTRACE_MCOUNT_MAX_OFFSET 3635 + /* 3636 + * Weak functions can still have an mcount/fentry that is saved in 3637 + * the __mcount_loc section. These can be detected by having a 3638 + * symbol offset of greater than FTRACE_MCOUNT_MAX_OFFSET, as the 3639 + * symbol found by kallsyms is not the function that the mcount/fentry 3640 + * is part of. The offset is much greater in these cases. 3641 + * 3642 + * Test the record to make sure that the ip points to a valid kallsyms 3643 + * and if not, mark it disabled. 3644 + */ 3645 + static int test_for_valid_rec(struct dyn_ftrace *rec) 3646 + { 3647 + char str[KSYM_SYMBOL_LEN]; 3648 + unsigned long offset; 3649 + const char *ret; 3650 + 3651 + ret = kallsyms_lookup(rec->ip, NULL, &offset, NULL, str); 3652 + 3653 + /* Weak functions can cause invalid addresses */ 3654 + if (!ret || offset > FTRACE_MCOUNT_MAX_OFFSET) { 3655 + rec->flags |= FTRACE_FL_DISABLED; 3656 + return 0; 3657 + } 3658 + return 1; 3659 + } 3660 + 3661 + static struct workqueue_struct *ftrace_check_wq __initdata; 3662 + static struct work_struct ftrace_check_work __initdata; 3663 + 3664 + /* 3665 + * Scan all the mcount/fentry entries to make sure they are valid. 3666 + */ 3667 + static __init void ftrace_check_work_func(struct work_struct *work) 3668 + { 3669 + struct ftrace_page *pg; 3670 + struct dyn_ftrace *rec; 3671 + 3672 + mutex_lock(&ftrace_lock); 3673 + do_for_each_ftrace_rec(pg, rec) { 3674 + test_for_valid_rec(rec); 3675 + } while_for_each_ftrace_rec(); 3676 + mutex_unlock(&ftrace_lock); 3677 + } 3678 + 3679 + static int __init ftrace_check_for_weak_functions(void) 3680 + { 3681 + INIT_WORK(&ftrace_check_work, ftrace_check_work_func); 3682 + 3683 + ftrace_check_wq = alloc_workqueue("ftrace_check_wq", WQ_UNBOUND, 0); 3684 + 3685 + queue_work(ftrace_check_wq, &ftrace_check_work); 3686 + return 0; 3687 + } 3688 + 3689 + static int __init ftrace_check_sync(void) 3690 + { 3691 + /* Make sure the ftrace_check updates are finished */ 3692 + if (ftrace_check_wq) 3693 + destroy_workqueue(ftrace_check_wq); 3694 + return 0; 3695 + } 3696 + 3697 + late_initcall_sync(ftrace_check_sync); 3698 + subsys_initcall(ftrace_check_for_weak_functions); 3699 + 3700 + static int print_rec(struct seq_file *m, unsigned long ip) 3701 + { 3702 + unsigned long offset; 3703 + char str[KSYM_SYMBOL_LEN]; 3704 + char *modname; 3705 + const char *ret; 3706 + 3707 + ret = kallsyms_lookup(ip, NULL, &offset, &modname, str); 3708 + /* Weak functions can cause invalid addresses */ 3709 + if (!ret || offset > FTRACE_MCOUNT_MAX_OFFSET) { 3710 + snprintf(str, KSYM_SYMBOL_LEN, "%s_%ld", 3711 + FTRACE_INVALID_FUNCTION, offset); 3712 + ret = NULL; 3713 + } 3714 + 3715 + seq_puts(m, str); 3716 + if (modname) 3717 + seq_printf(m, " [%s]", modname); 3718 + return ret == NULL ? -1 : 0; 3719 + } 3720 + #else 3721 + static inline int test_for_valid_rec(struct dyn_ftrace *rec) 3722 + { 3723 + return 1; 3724 + } 3725 + 3726 + static inline int print_rec(struct seq_file *m, unsigned long ip) 3727 + { 3728 + seq_printf(m, "%ps", (void *)ip); 3729 + return 0; 3730 + } 3731 + #endif 3732 + 3625 3733 static int t_show(struct seq_file *m, void *v) 3626 3734 { 3627 3735 struct ftrace_iterator *iter = m->private; ··· 3745 3655 if (!rec) 3746 3656 return 0; 3747 3657 3748 - seq_printf(m, "%ps", (void *)rec->ip); 3658 + if (print_rec(m, rec->ip)) { 3659 + /* This should only happen when a rec is disabled */ 3660 + WARN_ON_ONCE(!(rec->flags & FTRACE_FL_DISABLED)); 3661 + seq_putc(m, '\n'); 3662 + return 0; 3663 + } 3664 + 3749 3665 if (iter->flags & FTRACE_ITER_ENABLED) { 3750 3666 struct ftrace_ops *ops; 3751 3667 ··· 4069 3973 return 0; 4070 3974 } 4071 3975 3976 + #ifdef FTRACE_MCOUNT_MAX_OFFSET 3977 + static int lookup_ip(unsigned long ip, char **modname, char *str) 3978 + { 3979 + unsigned long offset; 3980 + 3981 + kallsyms_lookup(ip, NULL, &offset, modname, str); 3982 + if (offset > FTRACE_MCOUNT_MAX_OFFSET) 3983 + return -1; 3984 + return 0; 3985 + } 3986 + #else 3987 + static int lookup_ip(unsigned long ip, char **modname, char *str) 3988 + { 3989 + kallsyms_lookup(ip, NULL, NULL, modname, str); 3990 + return 0; 3991 + } 3992 + #endif 3993 + 4072 3994 static int 4073 3995 ftrace_match_record(struct dyn_ftrace *rec, struct ftrace_glob *func_g, 4074 3996 struct ftrace_glob *mod_g, int exclude_mod) ··· 4094 3980 char str[KSYM_SYMBOL_LEN]; 4095 3981 char *modname; 4096 3982 4097 - kallsyms_lookup(rec->ip, NULL, NULL, &modname, str); 3983 + if (lookup_ip(rec->ip, &modname, str)) { 3984 + /* This should only happen when a rec is disabled */ 3985 + WARN_ON_ONCE(system_state == SYSTEM_RUNNING && 3986 + !(rec->flags & FTRACE_FL_DISABLED)); 3987 + return 0; 3988 + } 4098 3989 4099 3990 if (mod_g) { 4100 3991 int mod_matches = (modname) ? ftrace_match(modname, mod_g) : 0; ··· 4550 4431 * @ip: The instruction pointer address to remove the data from 4551 4432 * 4552 4433 * Returns the data if it is found, otherwise NULL. 4553 - * Note, if the data pointer is used as the data itself, (see 4434 + * Note, if the data pointer is used as the data itself, (see 4554 4435 * ftrace_func_mapper_find_ip(), then the return value may be meaningless, 4555 4436 * if the data pointer was set to zero. 4556 4437 */ ··· 4645 4526 struct ftrace_probe_ops *probe_ops, 4646 4527 void *data) 4647 4528 { 4529 + struct ftrace_func_probe *probe = NULL, *iter; 4648 4530 struct ftrace_func_entry *entry; 4649 - struct ftrace_func_probe *probe; 4650 4531 struct ftrace_hash **orig_hash; 4651 4532 struct ftrace_hash *old_hash; 4652 4533 struct ftrace_hash *hash; ··· 4665 4546 4666 4547 mutex_lock(&ftrace_lock); 4667 4548 /* Check if the probe_ops is already registered */ 4668 - list_for_each_entry(probe, &tr->func_probes, list) { 4669 - if (probe->probe_ops == probe_ops) 4549 + list_for_each_entry(iter, &tr->func_probes, list) { 4550 + if (iter->probe_ops == probe_ops) { 4551 + probe = iter; 4670 4552 break; 4553 + } 4671 4554 } 4672 - if (&probe->list == &tr->func_probes) { 4555 + if (!probe) { 4673 4556 probe = kzalloc(sizeof(*probe), GFP_KERNEL); 4674 4557 if (!probe) { 4675 4558 mutex_unlock(&ftrace_lock); ··· 4789 4668 unregister_ftrace_function_probe_func(char *glob, struct trace_array *tr, 4790 4669 struct ftrace_probe_ops *probe_ops) 4791 4670 { 4671 + struct ftrace_func_probe *probe = NULL, *iter; 4792 4672 struct ftrace_ops_hash old_hash_ops; 4793 4673 struct ftrace_func_entry *entry; 4794 - struct ftrace_func_probe *probe; 4795 4674 struct ftrace_glob func_g; 4796 4675 struct ftrace_hash **orig_hash; 4797 4676 struct ftrace_hash *old_hash; ··· 4819 4698 4820 4699 mutex_lock(&ftrace_lock); 4821 4700 /* Check if the probe_ops is already registered */ 4822 - list_for_each_entry(probe, &tr->func_probes, list) { 4823 - if (probe->probe_ops == probe_ops) 4701 + list_for_each_entry(iter, &tr->func_probes, list) { 4702 + if (iter->probe_ops == probe_ops) { 4703 + probe = iter; 4824 4704 break; 4705 + } 4825 4706 } 4826 - if (&probe->list == &tr->func_probes) 4707 + if (!probe) 4827 4708 goto err_unlock_ftrace; 4828 4709 4829 4710 ret = -EINVAL; ··· 5284 5161 goto out_unlock; 5285 5162 5286 5163 ret = ftrace_set_filter_ip(&direct_ops, ip, 0, 0); 5287 - if (ret) 5288 - remove_hash_entry(direct_functions, entry); 5289 5164 5290 5165 if (!ret && !(direct_ops.flags & FTRACE_OPS_FL_ENABLED)) { 5291 5166 ret = register_ftrace_function(&direct_ops); ··· 5292 5171 } 5293 5172 5294 5173 if (ret) { 5174 + remove_hash_entry(direct_functions, entry); 5295 5175 kfree(entry); 5296 5176 if (!direct->count) { 5297 5177 list_del_rcu(&direct->next); ··· 6915 6793 !within_module_init(rec->ip, mod)) 6916 6794 break; 6917 6795 6796 + /* Weak functions should still be ignored */ 6797 + if (!test_for_valid_rec(rec)) { 6798 + /* Clear all other flags. Should not be enabled anyway */ 6799 + rec->flags = FTRACE_FL_DISABLED; 6800 + continue; 6801 + } 6802 + 6918 6803 cnt = 0; 6919 6804 6920 6805 /* ··· 6958 6829 6959 6830 void ftrace_module_init(struct module *mod) 6960 6831 { 6832 + int ret; 6833 + 6961 6834 if (ftrace_disabled || !mod->num_ftrace_callsites) 6962 6835 return; 6963 6836 6964 - ftrace_process_locs(mod, mod->ftrace_callsites, 6965 - mod->ftrace_callsites + mod->num_ftrace_callsites); 6837 + ret = ftrace_process_locs(mod, mod->ftrace_callsites, 6838 + mod->ftrace_callsites + mod->num_ftrace_callsites); 6839 + if (ret) 6840 + pr_warn("ftrace: failed to allocate entries for module '%s' functions\n", 6841 + mod->name); 6966 6842 } 6967 6843 6968 6844 static void save_ftrace_mod_rec(struct ftrace_mod_map *mod_map, ··· 7300 7166 pr_info("ftrace: allocating %ld entries in %ld pages\n", 7301 7167 count, count / ENTRIES_PER_PAGE + 1); 7302 7168 7303 - last_ftrace_enabled = ftrace_enabled = 1; 7304 - 7305 7169 ret = ftrace_process_locs(NULL, 7306 7170 __start_mcount_loc, 7307 7171 __stop_mcount_loc); 7172 + if (ret) { 7173 + pr_warn("ftrace: failed to allocate entries for functions\n"); 7174 + goto failed; 7175 + } 7308 7176 7309 7177 pr_info("ftrace: allocated %ld pages with %ld groups\n", 7310 7178 ftrace_number_of_pages, ftrace_number_of_groups); 7179 + 7180 + last_ftrace_enabled = ftrace_enabled = 1; 7311 7181 7312 7182 set_ftrace_early_filters(); 7313 7183
+2 -2
kernel/trace/pid_list.c
··· 118 118 /** 119 119 * trace_pid_list_is_set - test if the pid is set in the list 120 120 * @pid_list: The pid list to test 121 - * @pid: The pid to to see if set in the list. 121 + * @pid: The pid to see if set in the list. 122 122 * 123 - * Tests if @pid is is set in the @pid_list. This is usually called 123 + * Tests if @pid is set in the @pid_list. This is usually called 124 124 * from the scheduler when a task is scheduled. Its pid is checked 125 125 * if it should be traced or not. 126 126 *
+64 -17
kernel/trace/ring_buffer.c
··· 29 29 30 30 #include <asm/local.h> 31 31 32 + /* 33 + * The "absolute" timestamp in the buffer is only 59 bits. 34 + * If a clock has the 5 MSBs set, it needs to be saved and 35 + * reinserted. 36 + */ 37 + #define TS_MSB (0xf8ULL << 56) 38 + #define ABS_TS_MASK (~TS_MSB) 39 + 32 40 static void update_pages_handler(struct work_struct *work); 33 41 34 42 /* ··· 476 468 local_t cnt; 477 469 local_t top; 478 470 local_t bottom; 471 + local_t msb; 479 472 }; 480 473 #else 481 474 #include <asm/local64.h> ··· 578 569 * For the ring buffer, 64 bit required operations for the time is 579 570 * the following: 580 571 * 581 - * - Only need 59 bits (uses 60 to make it even). 582 572 * - Reads may fail if it interrupted a modification of the time stamp. 583 573 * It will succeed if it did not interrupt another write even if 584 574 * the read itself is interrupted by a write. ··· 602 594 */ 603 595 #define RB_TIME_SHIFT 30 604 596 #define RB_TIME_VAL_MASK ((1 << RB_TIME_SHIFT) - 1) 597 + #define RB_TIME_MSB_SHIFT 60 605 598 606 599 static inline int rb_time_cnt(unsigned long val) 607 600 { ··· 622 613 623 614 static inline bool __rb_time_read(rb_time_t *t, u64 *ret, unsigned long *cnt) 624 615 { 625 - unsigned long top, bottom; 616 + unsigned long top, bottom, msb; 626 617 unsigned long c; 627 618 628 619 /* ··· 634 625 c = local_read(&t->cnt); 635 626 top = local_read(&t->top); 636 627 bottom = local_read(&t->bottom); 628 + msb = local_read(&t->msb); 637 629 } while (c != local_read(&t->cnt)); 638 630 639 631 *cnt = rb_time_cnt(top); ··· 643 633 if (*cnt != rb_time_cnt(bottom)) 644 634 return false; 645 635 646 - *ret = rb_time_val(top, bottom); 636 + /* The shift to msb will lose its cnt bits */ 637 + *ret = rb_time_val(top, bottom) | ((u64)msb << RB_TIME_MSB_SHIFT); 647 638 return true; 648 639 } 649 640 ··· 660 649 return (val & RB_TIME_VAL_MASK) | ((cnt & 3) << RB_TIME_SHIFT); 661 650 } 662 651 663 - static inline void rb_time_split(u64 val, unsigned long *top, unsigned long *bottom) 652 + static inline void rb_time_split(u64 val, unsigned long *top, unsigned long *bottom, 653 + unsigned long *msb) 664 654 { 665 655 *top = (unsigned long)((val >> RB_TIME_SHIFT) & RB_TIME_VAL_MASK); 666 656 *bottom = (unsigned long)(val & RB_TIME_VAL_MASK); 657 + *msb = (unsigned long)(val >> RB_TIME_MSB_SHIFT); 667 658 } 668 659 669 660 static inline void rb_time_val_set(local_t *t, unsigned long val, unsigned long cnt) ··· 676 663 677 664 static void rb_time_set(rb_time_t *t, u64 val) 678 665 { 679 - unsigned long cnt, top, bottom; 666 + unsigned long cnt, top, bottom, msb; 680 667 681 - rb_time_split(val, &top, &bottom); 668 + rb_time_split(val, &top, &bottom, &msb); 682 669 683 670 /* Writes always succeed with a valid number even if it gets interrupted. */ 684 671 do { 685 672 cnt = local_inc_return(&t->cnt); 686 673 rb_time_val_set(&t->top, top, cnt); 687 674 rb_time_val_set(&t->bottom, bottom, cnt); 675 + rb_time_val_set(&t->msb, val >> RB_TIME_MSB_SHIFT, cnt); 688 676 } while (cnt != local_read(&t->cnt)); 689 677 } 690 678 ··· 700 686 701 687 static int rb_time_cmpxchg(rb_time_t *t, u64 expect, u64 set) 702 688 { 703 - unsigned long cnt, top, bottom; 704 - unsigned long cnt2, top2, bottom2; 689 + unsigned long cnt, top, bottom, msb; 690 + unsigned long cnt2, top2, bottom2, msb2; 705 691 u64 val; 706 692 707 693 /* The cmpxchg always fails if it interrupted an update */ ··· 717 703 718 704 cnt2 = cnt + 1; 719 705 720 - rb_time_split(val, &top, &bottom); 706 + rb_time_split(val, &top, &bottom, &msb); 721 707 top = rb_time_val_cnt(top, cnt); 722 708 bottom = rb_time_val_cnt(bottom, cnt); 723 709 724 - rb_time_split(set, &top2, &bottom2); 710 + rb_time_split(set, &top2, &bottom2, &msb2); 725 711 top2 = rb_time_val_cnt(top2, cnt2); 726 712 bottom2 = rb_time_val_cnt(bottom2, cnt2); 727 713 728 714 if (!rb_time_read_cmpxchg(&t->cnt, cnt, cnt2)) 715 + return false; 716 + if (!rb_time_read_cmpxchg(&t->msb, msb, msb2)) 729 717 return false; 730 718 if (!rb_time_read_cmpxchg(&t->top, top, top2)) 731 719 return false; ··· 799 783 } 800 784 #endif 801 785 786 + /* 787 + * The absolute time stamp drops the 5 MSBs and some clocks may 788 + * require them. The rb_fix_abs_ts() will take a previous full 789 + * time stamp, and add the 5 MSB of that time stamp on to the 790 + * saved absolute time stamp. Then they are compared in case of 791 + * the unlikely event that the latest time stamp incremented 792 + * the 5 MSB. 793 + */ 794 + static inline u64 rb_fix_abs_ts(u64 abs, u64 save_ts) 795 + { 796 + if (save_ts & TS_MSB) { 797 + abs |= save_ts & TS_MSB; 798 + /* Check for overflow */ 799 + if (unlikely(abs < save_ts)) 800 + abs += 1ULL << 59; 801 + } 802 + return abs; 803 + } 802 804 803 805 static inline u64 rb_time_stamp(struct trace_buffer *buffer); 804 806 ··· 845 811 u64 ts; 846 812 847 813 /* If the event includes an absolute time, then just use that */ 848 - if (event->type_len == RINGBUF_TYPE_TIME_STAMP) 849 - return rb_event_time_stamp(event); 814 + if (event->type_len == RINGBUF_TYPE_TIME_STAMP) { 815 + ts = rb_event_time_stamp(event); 816 + return rb_fix_abs_ts(ts, cpu_buffer->tail_page->page->time_stamp); 817 + } 850 818 851 819 nest = local_read(&cpu_buffer->committing); 852 820 verify_event(cpu_buffer, event); ··· 2790 2754 (RB_ADD_STAMP_FORCE | RB_ADD_STAMP_ABSOLUTE); 2791 2755 2792 2756 if (unlikely(info->delta > (1ULL << 59))) { 2757 + /* 2758 + * Some timers can use more than 59 bits, and when a timestamp 2759 + * is added to the buffer, it will lose those bits. 2760 + */ 2761 + if (abs && (info->ts & TS_MSB)) { 2762 + info->delta &= ABS_TS_MASK; 2763 + 2793 2764 /* did the clock go backwards */ 2794 - if (info->before == info->after && info->before > info->ts) { 2765 + } else if (info->before == info->after && info->before > info->ts) { 2795 2766 /* not interrupted */ 2796 2767 static int once; 2797 2768 ··· 3347 3304 3348 3305 case RINGBUF_TYPE_TIME_STAMP: 3349 3306 delta = rb_event_time_stamp(event); 3350 - ts = delta; 3307 + ts = rb_fix_abs_ts(delta, ts); 3351 3308 pr_warn(" [%lld] absolute:%lld TIME STAMP\n", ts, delta); 3352 3309 break; 3353 3310 ··· 3423 3380 3424 3381 case RINGBUF_TYPE_TIME_STAMP: 3425 3382 delta = rb_event_time_stamp(event); 3426 - ts = delta; 3383 + ts = rb_fix_abs_ts(delta, ts); 3427 3384 break; 3428 3385 3429 3386 case RINGBUF_TYPE_PADDING: ··· 4410 4367 4411 4368 case RINGBUF_TYPE_TIME_STAMP: 4412 4369 delta = rb_event_time_stamp(event); 4370 + delta = rb_fix_abs_ts(delta, cpu_buffer->read_stamp); 4413 4371 cpu_buffer->read_stamp = delta; 4414 4372 return; 4415 4373 ··· 4441 4397 4442 4398 case RINGBUF_TYPE_TIME_STAMP: 4443 4399 delta = rb_event_time_stamp(event); 4400 + delta = rb_fix_abs_ts(delta, iter->read_stamp); 4444 4401 iter->read_stamp = delta; 4445 4402 return; 4446 4403 ··· 4695 4650 case RINGBUF_TYPE_TIME_STAMP: 4696 4651 if (ts) { 4697 4652 *ts = rb_event_time_stamp(event); 4653 + *ts = rb_fix_abs_ts(*ts, reader->page->time_stamp); 4698 4654 ring_buffer_normalize_time_stamp(cpu_buffer->buffer, 4699 4655 cpu_buffer->cpu, ts); 4700 4656 } ··· 4787 4741 case RINGBUF_TYPE_TIME_STAMP: 4788 4742 if (ts) { 4789 4743 *ts = rb_event_time_stamp(event); 4744 + *ts = rb_fix_abs_ts(*ts, iter->head_page->page->time_stamp); 4790 4745 ring_buffer_normalize_time_stamp(cpu_buffer->buffer, 4791 4746 cpu_buffer->cpu, ts); 4792 4747 } ··· 6058 6011 pr_info(" total events: %ld\n", total_lost + total_read); 6059 6012 pr_info(" recorded len bytes: %ld\n", total_len); 6060 6013 pr_info(" recorded size bytes: %ld\n", total_size); 6061 - if (total_lost) 6014 + if (total_lost) { 6062 6015 pr_info(" With dropped events, record len and size may not match\n" 6063 6016 " alloced and written from above\n"); 6064 - if (!total_lost) { 6017 + } else { 6065 6018 if (RB_WARN_ON(buffer, total_len != total_alloc || 6066 6019 total_size != total_written)) 6067 6020 break;
+45 -27
kernel/trace/trace.c
··· 721 721 pos = 0; 722 722 723 723 ret = trace_get_user(&parser, ubuf, cnt, &pos); 724 - if (ret < 0 || !trace_parser_loaded(&parser)) 724 + if (ret < 0) 725 725 break; 726 726 727 727 read += ret; 728 728 ubuf += ret; 729 729 cnt -= ret; 730 + 731 + if (!trace_parser_loaded(&parser)) 732 + break; 730 733 731 734 ret = -EINVAL; 732 735 if (kstrtoul(parser.buffer, 0, &val)) ··· 756 753 if (!nr_pids) { 757 754 /* Cleared the list of pids */ 758 755 trace_pid_list_free(pid_list); 759 - read = ret; 760 756 pid_list = NULL; 761 757 } 762 758 ··· 1176 1174 EXPORT_SYMBOL_GPL(tracing_snapshot_cond); 1177 1175 1178 1176 /** 1179 - * tracing_snapshot_cond_data - get the user data associated with a snapshot 1177 + * tracing_cond_snapshot_data - get the user data associated with a snapshot 1180 1178 * @tr: The tracing instance 1181 1179 * 1182 1180 * When the user enables a conditional snapshot using ··· 1544 1542 { ktime_get_mono_fast_ns, "mono", 1 }, 1545 1543 { ktime_get_raw_fast_ns, "mono_raw", 1 }, 1546 1544 { ktime_get_boot_fast_ns, "boot", 1 }, 1545 + { ktime_get_tai_fast_ns, "tai", 1 }, 1547 1546 ARCH_TRACE_CLOCKS 1548 1547 }; 1549 1548 ··· 2838 2835 } 2839 2836 EXPORT_SYMBOL_GPL(trace_event_buffer_lock_reserve); 2840 2837 2841 - static DEFINE_SPINLOCK(tracepoint_iter_lock); 2838 + static DEFINE_RAW_SPINLOCK(tracepoint_iter_lock); 2842 2839 static DEFINE_MUTEX(tracepoint_printk_mutex); 2843 2840 2844 2841 static void output_printk(struct trace_event_buffer *fbuffer) ··· 2866 2863 2867 2864 event = &fbuffer->trace_file->event_call->event; 2868 2865 2869 - spin_lock_irqsave(&tracepoint_iter_lock, flags); 2866 + raw_spin_lock_irqsave(&tracepoint_iter_lock, flags); 2870 2867 trace_seq_init(&iter->seq); 2871 2868 iter->ent = fbuffer->entry; 2872 2869 event_call->event.funcs->trace(iter, 0, event); 2873 2870 trace_seq_putc(&iter->seq, 0); 2874 2871 printk("%s", iter->seq.buffer); 2875 2872 2876 - spin_unlock_irqrestore(&tracepoint_iter_lock, flags); 2873 + raw_spin_unlock_irqrestore(&tracepoint_iter_lock, flags); 2877 2874 } 2878 2875 2879 2876 int tracepoint_printk_sysctl(struct ctl_table *table, int write, ··· 4252 4249 unsigned int flags) 4253 4250 { 4254 4251 bool tgid = flags & TRACE_ITER_RECORD_TGID; 4255 - const char *space = " "; 4252 + static const char space[] = " "; 4256 4253 int prec = tgid ? 12 : 2; 4257 4254 4258 4255 print_event_info(buf, m); ··· 4276 4273 struct tracer *type = iter->trace; 4277 4274 unsigned long entries; 4278 4275 unsigned long total; 4279 - const char *name = "preemption"; 4280 - 4281 - name = type->name; 4276 + const char *name = type->name; 4282 4277 4283 4278 get_total_entries(buf, &total, &entries); 4284 4279 ··· 5470 5469 " error_log\t- error log for failed commands (that support it)\n" 5471 5470 " buffer_size_kb\t- view and modify size of per cpu buffer\n" 5472 5471 " buffer_total_size_kb - view total size of all cpu buffers\n\n" 5473 - " trace_clock\t\t-change the clock used to order events\n" 5472 + " trace_clock\t\t- change the clock used to order events\n" 5474 5473 " local: Per cpu clock but may not be synced across CPUs\n" 5475 5474 " global: Synced across CPUs but slows tracing down.\n" 5476 5475 " counter: Not a clock, but just an increment\n" ··· 5479 5478 #ifdef CONFIG_X86_64 5480 5479 " x86-tsc: TSC cycle counter\n" 5481 5480 #endif 5482 - "\n timestamp_mode\t-view the mode used to timestamp events\n" 5481 + "\n timestamp_mode\t- view the mode used to timestamp events\n" 5483 5482 " delta: Delta difference against a buffer-wide timestamp\n" 5484 5483 " absolute: Absolute (standalone) timestamp\n" 5485 5484 "\n trace_marker\t\t- Writes into this file writes into the kernel buffer\n" ··· 6327 6326 tr->current_trace = &nop_trace; 6328 6327 } 6329 6328 6329 + static bool tracer_options_updated; 6330 + 6330 6331 static void add_tracer_options(struct trace_array *tr, struct tracer *t) 6331 6332 { 6332 6333 /* Only enable if the directory has been created already. */ 6333 6334 if (!tr->dir) 6335 + return; 6336 + 6337 + /* Only create trace option files after update_tracer_options finish */ 6338 + if (!tracer_options_updated) 6334 6339 return; 6335 6340 6336 6341 create_trace_option_files(tr, t); ··· 6455 6448 { 6456 6449 struct trace_array *tr = filp->private_data; 6457 6450 char buf[MAX_TRACER_SIZE+1]; 6458 - int i; 6451 + char *name; 6459 6452 size_t ret; 6460 6453 int err; 6461 6454 ··· 6469 6462 6470 6463 buf[cnt] = 0; 6471 6464 6472 - /* strip ending whitespace. */ 6473 - for (i = cnt - 1; i > 0 && isspace(buf[i]); i--) 6474 - buf[i] = 0; 6465 + name = strim(buf); 6475 6466 6476 - err = tracing_set_tracer(tr, buf); 6467 + err = tracing_set_tracer(tr, name); 6477 6468 if (err) 6478 6469 return err; 6479 6470 ··· 9175 9170 static void update_tracer_options(struct trace_array *tr) 9176 9171 { 9177 9172 mutex_lock(&trace_types_lock); 9173 + tracer_options_updated = true; 9178 9174 __update_tracer_options(tr); 9179 9175 mutex_unlock(&trace_types_lock); 9180 9176 } ··· 9608 9602 9609 9603 static struct workqueue_struct *eval_map_wq __initdata; 9610 9604 static struct work_struct eval_map_work __initdata; 9605 + static struct work_struct tracerfs_init_work __initdata; 9611 9606 9612 9607 static void __init eval_map_work_func(struct work_struct *work) 9613 9608 { ··· 9633 9626 queue_work(eval_map_wq, &eval_map_work); 9634 9627 return 0; 9635 9628 } 9629 + 9630 + subsys_initcall(trace_eval_init); 9636 9631 9637 9632 static int __init trace_eval_sync(void) 9638 9633 { ··· 9718 9709 }; 9719 9710 #endif /* CONFIG_MODULES */ 9720 9711 9721 - static __init int tracer_init_tracefs(void) 9712 + static __init void tracer_init_tracefs_work_func(struct work_struct *work) 9722 9713 { 9723 - int ret; 9724 - 9725 - trace_access_lock_init(); 9726 - 9727 - ret = tracing_init_dentry(); 9728 - if (ret) 9729 - return 0; 9730 9714 9731 9715 event_trace_init(); 9732 9716 ··· 9741 9739 trace_create_file("saved_tgids", TRACE_MODE_READ, NULL, 9742 9740 NULL, &tracing_saved_tgids_fops); 9743 9741 9744 - trace_eval_init(); 9745 - 9746 9742 trace_create_eval_file(NULL); 9747 9743 9748 9744 #ifdef CONFIG_MODULES ··· 9755 9755 create_trace_instances(NULL); 9756 9756 9757 9757 update_tracer_options(&global_trace); 9758 + } 9759 + 9760 + static __init int tracer_init_tracefs(void) 9761 + { 9762 + int ret; 9763 + 9764 + trace_access_lock_init(); 9765 + 9766 + ret = tracing_init_dentry(); 9767 + if (ret) 9768 + return 0; 9769 + 9770 + if (eval_map_wq) { 9771 + INIT_WORK(&tracerfs_init_work, tracer_init_tracefs_work_func); 9772 + queue_work(eval_map_wq, &tracerfs_init_work); 9773 + } else { 9774 + tracer_init_tracefs_work_func(NULL); 9775 + } 9758 9776 9759 9777 return 0; 9760 9778 }
+11 -15
kernel/trace/trace.h
··· 1573 1573 }; 1574 1574 1575 1575 extern int event_enable_trigger_print(struct seq_file *m, 1576 - struct event_trigger_ops *ops, 1577 1576 struct event_trigger_data *data); 1578 - extern void event_enable_trigger_free(struct event_trigger_ops *ops, 1579 - struct event_trigger_data *data); 1577 + extern void event_enable_trigger_free(struct event_trigger_data *data); 1580 1578 extern int event_enable_trigger_parse(struct event_command *cmd_ops, 1581 1579 struct trace_event_file *file, 1582 - char *glob, char *cmd, char *param); 1580 + char *glob, char *cmd, 1581 + char *param_and_filter); 1583 1582 extern int event_enable_register_trigger(char *glob, 1584 1583 struct event_trigger_data *data, 1585 1584 struct trace_event_file *file); ··· 1586 1587 struct event_trigger_data *test, 1587 1588 struct trace_event_file *file); 1588 1589 extern void trigger_data_free(struct event_trigger_data *data); 1589 - extern int event_trigger_init(struct event_trigger_ops *ops, 1590 - struct event_trigger_data *data); 1590 + extern int event_trigger_init(struct event_trigger_data *data); 1591 1591 extern int trace_event_trigger_enable_disable(struct trace_event_file *file, 1592 1592 int trigger_enable); 1593 1593 extern void update_cond_flag(struct trace_event_file *file); ··· 1627 1629 extern int event_trigger_register(struct event_command *cmd_ops, 1628 1630 struct trace_event_file *file, 1629 1631 char *glob, 1630 - char *cmd, 1631 - char *trigger, 1632 - struct event_trigger_data *trigger_data, 1633 - int *n_registered); 1632 + struct event_trigger_data *trigger_data); 1633 + extern void event_trigger_unregister(struct event_command *cmd_ops, 1634 + struct trace_event_file *file, 1635 + char *glob, 1636 + struct event_trigger_data *trigger_data); 1634 1637 1635 1638 /** 1636 1639 * struct event_trigger_ops - callbacks for trace event triggers ··· 1685 1686 struct trace_buffer *buffer, 1686 1687 void *rec, 1687 1688 struct ring_buffer_event *rbe); 1688 - int (*init)(struct event_trigger_ops *ops, 1689 - struct event_trigger_data *data); 1690 - void (*free)(struct event_trigger_ops *ops, 1691 - struct event_trigger_data *data); 1689 + int (*init)(struct event_trigger_data *data); 1690 + void (*free)(struct event_trigger_data *data); 1692 1691 int (*print)(struct seq_file *m, 1693 - struct event_trigger_ops *ops, 1694 1692 struct event_trigger_data *data); 1695 1693 }; 1696 1694
+1 -1
kernel/trace/trace_boot.c
··· 300 300 { 301 301 struct xbc_node *node; 302 302 const char *p, *handler; 303 - int ret; 303 + int ret = 0; 304 304 305 305 handler = xbc_node_get_data(hnode); 306 306
+2 -7
kernel/trace/trace_dynevent.c
··· 255 255 /* Make a tracefs interface for controlling dynamic events */ 256 256 static __init int init_dynamic_event(void) 257 257 { 258 - struct dentry *entry; 259 258 int ret; 260 259 261 260 ret = tracing_init_dentry(); 262 261 if (ret) 263 262 return 0; 264 263 265 - entry = tracefs_create_file("dynamic_events", TRACE_MODE_WRITE, NULL, 266 - NULL, &dynamic_events_ops); 267 - 268 - /* Event list interface */ 269 - if (!entry) 270 - pr_warn("Could not create tracefs 'dynamic_events' entry\n"); 264 + trace_create_file("dynamic_events", TRACE_MODE_WRITE, NULL, 265 + NULL, &dynamic_events_ops); 271 266 272 267 return 0; 273 268 }
+12 -12
kernel/trace/trace_eprobe.c
··· 511 511 * functions are just stubs to fulfill what is needed to use the trigger 512 512 * infrastructure. 513 513 */ 514 - static int eprobe_trigger_init(struct event_trigger_ops *ops, 515 - struct event_trigger_data *data) 514 + static int eprobe_trigger_init(struct event_trigger_data *data) 516 515 { 517 516 return 0; 518 517 } 519 518 520 - static void eprobe_trigger_free(struct event_trigger_ops *ops, 521 - struct event_trigger_data *data) 519 + static void eprobe_trigger_free(struct event_trigger_data *data) 522 520 { 523 521 524 522 } 525 523 526 524 static int eprobe_trigger_print(struct seq_file *m, 527 - struct event_trigger_ops *ops, 528 525 struct event_trigger_data *data) 529 526 { 530 527 /* Do not print eprobe event triggers */ ··· 546 549 547 550 static int eprobe_trigger_cmd_parse(struct event_command *cmd_ops, 548 551 struct trace_event_file *file, 549 - char *glob, char *cmd, char *param) 552 + char *glob, char *cmd, 553 + char *param_and_filter) 550 554 { 551 555 return -1; 552 556 } ··· 648 650 static int disable_eprobe(struct trace_eprobe *ep, 649 651 struct trace_array *tr) 650 652 { 651 - struct event_trigger_data *trigger; 653 + struct event_trigger_data *trigger = NULL, *iter; 652 654 struct trace_event_file *file; 653 655 struct eprobe_data *edata; 654 656 ··· 656 658 if (!file) 657 659 return -ENOENT; 658 660 659 - list_for_each_entry(trigger, &file->triggers, list) { 660 - if (!(trigger->flags & EVENT_TRIGGER_FL_PROBE)) 661 + list_for_each_entry(iter, &file->triggers, list) { 662 + if (!(iter->flags & EVENT_TRIGGER_FL_PROBE)) 661 663 continue; 662 - edata = trigger->private_data; 663 - if (edata->ep == ep) 664 + edata = iter->private_data; 665 + if (edata->ep == ep) { 666 + trigger = iter; 664 667 break; 668 + } 665 669 } 666 - if (list_entry_is_head(trigger, &file->triggers, list)) 670 + if (!trigger) 667 671 return -ENODEV; 668 672 669 673 list_del_rcu(&trigger->list);
+30 -39
kernel/trace/trace_events.c
··· 392 392 if (!(dereference_flags & (1ULL << arg))) 393 393 goto next_arg; 394 394 395 - /* Check for __get_sockaddr */; 396 - if (str_has_prefix(fmt + i, "__get_sockaddr(")) { 397 - dereference_flags &= ~(1ULL << arg); 398 - goto next_arg; 399 - } 400 - 401 395 /* Find the REC-> in the argument */ 402 396 c = strchr(fmt + i, ','); 403 397 r = strstr(fmt + i, "REC->"); ··· 407 413 a = strchr(fmt + i, '&'); 408 414 if ((a && (a < r)) || test_field(r, call)) 409 415 dereference_flags &= ~(1ULL << arg); 416 + } else if ((r = strstr(fmt + i, "__get_dynamic_array(")) && 417 + (!c || r < c)) { 418 + dereference_flags &= ~(1ULL << arg); 419 + } else if ((r = strstr(fmt + i, "__get_sockaddr(")) && 420 + (!c || r < c)) { 421 + dereference_flags &= ~(1ULL << arg); 410 422 } 423 + 411 424 next_arg: 412 425 i--; 413 426 arg++; ··· 1724 1723 1725 1724 static int subsystem_open(struct inode *inode, struct file *filp) 1726 1725 { 1726 + struct trace_subsystem_dir *dir = NULL, *iter_dir; 1727 + struct trace_array *tr = NULL, *iter_tr; 1727 1728 struct event_subsystem *system = NULL; 1728 - struct trace_subsystem_dir *dir = NULL; /* Initialize for gcc */ 1729 - struct trace_array *tr; 1730 1729 int ret; 1731 1730 1732 1731 if (tracing_is_disabled()) ··· 1735 1734 /* Make sure the system still exists */ 1736 1735 mutex_lock(&event_mutex); 1737 1736 mutex_lock(&trace_types_lock); 1738 - list_for_each_entry(tr, &ftrace_trace_arrays, list) { 1739 - list_for_each_entry(dir, &tr->systems, list) { 1740 - if (dir == inode->i_private) { 1737 + list_for_each_entry(iter_tr, &ftrace_trace_arrays, list) { 1738 + list_for_each_entry(iter_dir, &iter_tr->systems, list) { 1739 + if (iter_dir == inode->i_private) { 1741 1740 /* Don't open systems with no events */ 1741 + tr = iter_tr; 1742 + dir = iter_dir; 1742 1743 if (dir->nr_events) { 1743 1744 __get_system_dir(dir); 1744 1745 system = dir->subsystem; ··· 1755 1752 1756 1753 if (!system) 1757 1754 return -ENODEV; 1758 - 1759 - /* Some versions of gcc think dir can be uninitialized here */ 1760 - WARN_ON(!dir); 1761 1755 1762 1756 /* Still need to increment the ref count of the system */ 1763 1757 if (trace_array_get(tr) < 0) { ··· 2280 2280 event_subsystem_dir(struct trace_array *tr, const char *name, 2281 2281 struct trace_event_file *file, struct dentry *parent) 2282 2282 { 2283 + struct event_subsystem *system, *iter; 2283 2284 struct trace_subsystem_dir *dir; 2284 - struct event_subsystem *system; 2285 2285 struct dentry *entry; 2286 2286 2287 2287 /* First see if we did not already create this dir */ ··· 2295 2295 } 2296 2296 2297 2297 /* Now see if the system itself exists. */ 2298 - list_for_each_entry(system, &event_subsystems, list) { 2299 - if (strcmp(system->name, name) == 0) 2298 + system = NULL; 2299 + list_for_each_entry(iter, &event_subsystems, list) { 2300 + if (strcmp(iter->name, name) == 0) { 2301 + system = iter; 2300 2302 break; 2303 + } 2301 2304 } 2302 - /* Reset system variable when not found */ 2303 - if (&system->list == &event_subsystems) 2304 - system = NULL; 2305 2305 2306 2306 dir = kmalloc(sizeof(*dir), GFP_KERNEL); 2307 2307 if (!dir) ··· 3546 3546 struct dentry *d_events; 3547 3547 struct dentry *entry; 3548 3548 3549 - entry = tracefs_create_file("set_event", TRACE_MODE_WRITE, parent, 3550 - tr, &ftrace_set_event_fops); 3551 - if (!entry) { 3552 - pr_warn("Could not create tracefs 'set_event' entry\n"); 3549 + entry = trace_create_file("set_event", TRACE_MODE_WRITE, parent, 3550 + tr, &ftrace_set_event_fops); 3551 + if (!entry) 3553 3552 return -ENOMEM; 3554 - } 3555 3553 3556 3554 d_events = tracefs_create_dir("events", parent); 3557 3555 if (!d_events) { ··· 3564 3566 3565 3567 /* There are not as crucial, just warn if they are not created */ 3566 3568 3567 - entry = tracefs_create_file("set_event_pid", TRACE_MODE_WRITE, parent, 3568 - tr, &ftrace_set_event_pid_fops); 3569 - if (!entry) 3570 - pr_warn("Could not create tracefs 'set_event_pid' entry\n"); 3569 + trace_create_file("set_event_pid", TRACE_MODE_WRITE, parent, 3570 + tr, &ftrace_set_event_pid_fops); 3571 3571 3572 - entry = tracefs_create_file("set_event_notrace_pid", 3573 - TRACE_MODE_WRITE, parent, tr, 3574 - &ftrace_set_event_notrace_pid_fops); 3575 - if (!entry) 3576 - pr_warn("Could not create tracefs 'set_event_notrace_pid' entry\n"); 3572 + trace_create_file("set_event_notrace_pid", 3573 + TRACE_MODE_WRITE, parent, tr, 3574 + &ftrace_set_event_notrace_pid_fops); 3577 3575 3578 3576 /* ring buffer internal formats */ 3579 3577 trace_create_file("header_page", TRACE_MODE_READ, d_events, ··· 3784 3790 __init int event_trace_init(void) 3785 3791 { 3786 3792 struct trace_array *tr; 3787 - struct dentry *entry; 3788 3793 int ret; 3789 3794 3790 3795 tr = top_trace_array(); 3791 3796 if (!tr) 3792 3797 return -ENODEV; 3793 3798 3794 - entry = tracefs_create_file("available_events", TRACE_MODE_READ, 3795 - NULL, tr, &ftrace_avail_fops); 3796 - if (!entry) 3797 - pr_warn("Could not create tracefs 'available_events' entry\n"); 3799 + trace_create_file("available_events", TRACE_MODE_READ, 3800 + NULL, tr, &ftrace_avail_fops); 3798 3801 3799 3802 ret = early_event_add_tracer(NULL, tr); 3800 3803 if (ret)
+1 -1
kernel/trace/trace_events_filter.c
··· 1816 1816 * create_filter - create a filter for a trace_event_call 1817 1817 * @tr: the trace array associated with these events 1818 1818 * @call: trace_event_call to create a filter for 1819 - * @filter_str: filter string 1819 + * @filter_string: filter string 1820 1820 * @set_str: remember @filter_str and enable detailed error in filter 1821 1821 * @filterp: out param for created filter (always updated on return) 1822 1822 * Must be a pointer that references a NULL pointer.
+102 -95
kernel/trace/trace_events_hist.c
··· 2093 2093 return err; 2094 2094 free: 2095 2095 kfree(ref_field->system); 2096 + ref_field->system = NULL; 2096 2097 kfree(ref_field->event_name); 2098 + ref_field->event_name = NULL; 2097 2099 kfree(ref_field->name); 2100 + ref_field->name = NULL; 2098 2101 2099 2102 goto out; 2100 2103 } ··· 2788 2785 static struct event_command trigger_hist_cmd; 2789 2786 static int event_hist_trigger_parse(struct event_command *cmd_ops, 2790 2787 struct trace_event_file *file, 2791 - char *glob, char *cmd, char *param); 2788 + char *glob, char *cmd, 2789 + char *param_and_filter); 2792 2790 2793 2791 static bool compatible_keys(struct hist_trigger_data *target_hist_data, 2794 2792 struct hist_trigger_data *hist_data, ··· 4165 4161 return __create_val_field(hist_data, val_idx, file, NULL, field_str, 0); 4166 4162 } 4167 4163 4168 - static const char *no_comm = "(no comm)"; 4164 + static const char no_comm[] = "(no comm)"; 4169 4165 4170 4166 static u64 hist_field_execname(struct hist_field *hist_field, 4171 4167 struct tracing_map_elt *elt, ··· 5256 5252 seq_puts(m, "\n\n"); 5257 5253 5258 5254 seq_puts(m, "# event histogram\n#\n# trigger info: "); 5259 - data->ops->print(m, data->ops, data); 5255 + data->ops->print(m, data); 5260 5256 seq_puts(m, "#\n\n"); 5261 5257 5262 5258 hist_data = data->private_data; ··· 5488 5484 seq_puts(m, "\n\n"); 5489 5485 5490 5486 seq_puts(m, "# event histogram\n#\n# trigger info: "); 5491 - data->ops->print(m, data->ops, data); 5487 + data->ops->print(m, data); 5492 5488 seq_puts(m, "#\n\n"); 5493 5489 5494 5490 hist_data = data->private_data; ··· 5625 5621 } 5626 5622 5627 5623 static int event_hist_trigger_print(struct seq_file *m, 5628 - struct event_trigger_ops *ops, 5629 5624 struct event_trigger_data *data) 5630 5625 { 5631 5626 struct hist_trigger_data *hist_data = data->private_data; ··· 5732 5729 return 0; 5733 5730 } 5734 5731 5735 - static int event_hist_trigger_init(struct event_trigger_ops *ops, 5736 - struct event_trigger_data *data) 5732 + static int event_hist_trigger_init(struct event_trigger_data *data) 5737 5733 { 5738 5734 struct hist_trigger_data *hist_data = data->private_data; 5739 5735 ··· 5760 5758 } 5761 5759 } 5762 5760 5763 - static void event_hist_trigger_free(struct event_trigger_ops *ops, 5764 - struct event_trigger_data *data) 5761 + static void event_hist_trigger_free(struct event_trigger_data *data) 5765 5762 { 5766 5763 struct hist_trigger_data *hist_data = data->private_data; 5767 5764 ··· 5789 5788 .free = event_hist_trigger_free, 5790 5789 }; 5791 5790 5792 - static int event_hist_trigger_named_init(struct event_trigger_ops *ops, 5793 - struct event_trigger_data *data) 5791 + static int event_hist_trigger_named_init(struct event_trigger_data *data) 5794 5792 { 5795 5793 data->ref++; 5796 5794 5797 5795 save_named_trigger(data->named_data->name, data); 5798 5796 5799 - event_hist_trigger_init(ops, data->named_data); 5797 + event_hist_trigger_init(data->named_data); 5800 5798 5801 5799 return 0; 5802 5800 } 5803 5801 5804 - static void event_hist_trigger_named_free(struct event_trigger_ops *ops, 5805 - struct event_trigger_data *data) 5802 + static void event_hist_trigger_named_free(struct event_trigger_data *data) 5806 5803 { 5807 5804 if (WARN_ON_ONCE(data->ref <= 0)) 5808 5805 return; 5809 5806 5810 - event_hist_trigger_free(ops, data->named_data); 5807 + event_hist_trigger_free(data->named_data); 5811 5808 5812 5809 data->ref--; 5813 5810 if (!data->ref) { ··· 5932 5933 return true; 5933 5934 } 5934 5935 5936 + static bool existing_hist_update_only(char *glob, 5937 + struct event_trigger_data *data, 5938 + struct trace_event_file *file) 5939 + { 5940 + struct hist_trigger_data *hist_data = data->private_data; 5941 + struct event_trigger_data *test, *named_data = NULL; 5942 + bool updated = false; 5943 + 5944 + if (!hist_data->attrs->pause && !hist_data->attrs->cont && 5945 + !hist_data->attrs->clear) 5946 + goto out; 5947 + 5948 + if (hist_data->attrs->name) { 5949 + named_data = find_named_trigger(hist_data->attrs->name); 5950 + if (named_data) { 5951 + if (!hist_trigger_match(data, named_data, named_data, 5952 + true)) 5953 + goto out; 5954 + } 5955 + } 5956 + 5957 + if (hist_data->attrs->name && !named_data) 5958 + goto out; 5959 + 5960 + list_for_each_entry(test, &file->triggers, list) { 5961 + if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) { 5962 + if (!hist_trigger_match(data, test, named_data, false)) 5963 + continue; 5964 + if (hist_data->attrs->pause) 5965 + test->paused = true; 5966 + else if (hist_data->attrs->cont) 5967 + test->paused = false; 5968 + else if (hist_data->attrs->clear) 5969 + hist_clear(test); 5970 + updated = true; 5971 + goto out; 5972 + } 5973 + } 5974 + out: 5975 + return updated; 5976 + } 5977 + 5935 5978 static int hist_register_trigger(char *glob, 5936 5979 struct event_trigger_data *data, 5937 5980 struct trace_event_file *file) ··· 6002 5961 6003 5962 list_for_each_entry(test, &file->triggers, list) { 6004 5963 if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) { 6005 - if (!hist_trigger_match(data, test, named_data, false)) 6006 - continue; 6007 - if (hist_data->attrs->pause) 6008 - test->paused = true; 6009 - else if (hist_data->attrs->cont) 6010 - test->paused = false; 6011 - else if (hist_data->attrs->clear) 6012 - hist_clear(test); 6013 - else { 5964 + if (hist_trigger_match(data, test, named_data, false)) { 6014 5965 hist_err(tr, HIST_ERR_TRIGGER_EEXIST, 0); 6015 5966 ret = -EEXIST; 5967 + goto out; 6016 5968 } 6017 - goto out; 6018 5969 } 6019 5970 } 6020 5971 new: ··· 6026 5993 } 6027 5994 6028 5995 if (data->ops->init) { 6029 - ret = data->ops->init(data->ops, data); 5996 + ret = data->ops->init(data); 6030 5997 if (ret < 0) 6031 5998 goto out; 6032 5999 } ··· 6045 6012 6046 6013 if (named_data) 6047 6014 destroy_hist_data(hist_data); 6048 - 6049 - ret++; 6050 6015 out: 6051 6016 return ret; 6052 6017 } ··· 6120 6089 struct event_trigger_data *data, 6121 6090 struct trace_event_file *file) 6122 6091 { 6092 + struct event_trigger_data *test = NULL, *iter, *named_data = NULL; 6123 6093 struct hist_trigger_data *hist_data = data->private_data; 6124 - struct event_trigger_data *test, *named_data = NULL; 6125 - bool unregistered = false; 6126 6094 6127 6095 lockdep_assert_held(&event_mutex); 6128 6096 6129 6097 if (hist_data->attrs->name) 6130 6098 named_data = find_named_trigger(hist_data->attrs->name); 6131 6099 6132 - list_for_each_entry(test, &file->triggers, list) { 6133 - if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) { 6134 - if (!hist_trigger_match(data, test, named_data, false)) 6100 + list_for_each_entry(iter, &file->triggers, list) { 6101 + if (iter->cmd_ops->trigger_type == ETT_EVENT_HIST) { 6102 + if (!hist_trigger_match(data, iter, named_data, false)) 6135 6103 continue; 6136 - unregistered = true; 6104 + test = iter; 6137 6105 list_del_rcu(&test->list); 6138 6106 trace_event_trigger_enable_disable(file, 0); 6139 6107 update_cond_flag(file); ··· 6140 6110 } 6141 6111 } 6142 6112 6143 - if (unregistered && test->ops->free) 6144 - test->ops->free(test->ops, test); 6113 + if (test && test->ops->free) 6114 + test->ops->free(test); 6145 6115 6146 6116 if (hist_data->enable_timestamps) { 6147 - if (!hist_data->remove || unregistered) 6117 + if (!hist_data->remove || test) 6148 6118 tracing_set_filter_buffering(file->tr, false); 6149 6119 } 6150 6120 } ··· 6194 6164 if (hist_data->enable_timestamps) 6195 6165 tracing_set_filter_buffering(file->tr, false); 6196 6166 if (test->ops->free) 6197 - test->ops->free(test->ops, test); 6167 + test->ops->free(test); 6198 6168 } 6199 6169 } 6200 6170 } 6201 6171 6202 6172 static int event_hist_trigger_parse(struct event_command *cmd_ops, 6203 6173 struct trace_event_file *file, 6204 - char *glob, char *cmd, char *param) 6174 + char *glob, char *cmd, 6175 + char *param_and_filter) 6205 6176 { 6206 6177 unsigned int hist_trigger_bits = TRACING_MAP_BITS_DEFAULT; 6207 6178 struct event_trigger_data *trigger_data; 6208 6179 struct hist_trigger_attrs *attrs; 6209 - struct event_trigger_ops *trigger_ops; 6210 6180 struct hist_trigger_data *hist_data; 6181 + char *param, *filter, *p, *start; 6211 6182 struct synth_event *se; 6212 6183 const char *se_name; 6213 - bool remove = false; 6214 - char *trigger, *p, *start; 6184 + bool remove; 6215 6185 int ret = 0; 6216 6186 6217 6187 lockdep_assert_held(&event_mutex); 6218 6188 6219 - WARN_ON(!glob); 6220 - 6221 - if (strlen(glob)) { 6222 - hist_err_clear(); 6223 - last_cmd_set(file, param); 6224 - } 6225 - 6226 - if (!param) 6189 + if (WARN_ON(!glob)) 6227 6190 return -EINVAL; 6228 6191 6229 - if (glob[0] == '!') 6230 - remove = true; 6192 + if (glob[0]) { 6193 + hist_err_clear(); 6194 + last_cmd_set(file, param_and_filter); 6195 + } 6196 + 6197 + remove = event_trigger_check_remove(glob); 6198 + 6199 + if (event_trigger_empty_param(param_and_filter)) 6200 + return -EINVAL; 6231 6201 6232 6202 /* 6233 6203 * separate the trigger from the filter (k:v [if filter]) 6234 6204 * allowing for whitespace in the trigger 6235 6205 */ 6236 - p = trigger = param; 6206 + p = param = param_and_filter; 6237 6207 do { 6238 6208 p = strstr(p, "if"); 6239 6209 if (!p) 6240 6210 break; 6241 - if (p == param) 6211 + if (p == param_and_filter) 6242 6212 return -EINVAL; 6243 6213 if (*(p - 1) != ' ' && *(p - 1) != '\t') { 6244 6214 p++; 6245 6215 continue; 6246 6216 } 6247 - if (p >= param + strlen(param) - (sizeof("if") - 1) - 1) 6217 + if (p >= param_and_filter + strlen(param_and_filter) - (sizeof("if") - 1) - 1) 6248 6218 return -EINVAL; 6249 6219 if (*(p + sizeof("if") - 1) != ' ' && *(p + sizeof("if") - 1) != '\t') { 6250 6220 p++; ··· 6254 6224 } while (1); 6255 6225 6256 6226 if (!p) 6257 - param = NULL; 6227 + filter = NULL; 6258 6228 else { 6259 6229 *(p - 1) = '\0'; 6260 - param = strstrip(p); 6261 - trigger = strstrip(trigger); 6230 + filter = strstrip(p); 6231 + param = strstrip(param); 6262 6232 } 6263 6233 6264 6234 /* 6265 6235 * To simplify arithmetic expression parsing, replace occurrences of 6266 6236 * '.sym-offset' modifier with '.symXoffset' 6267 6237 */ 6268 - start = strstr(trigger, ".sym-offset"); 6238 + start = strstr(param, ".sym-offset"); 6269 6239 while (start) { 6270 6240 *(start + 4) = 'X'; 6271 6241 start = strstr(start + 11, ".sym-offset"); 6272 6242 } 6273 6243 6274 - attrs = parse_hist_trigger_attrs(file->tr, trigger); 6244 + attrs = parse_hist_trigger_attrs(file->tr, param); 6275 6245 if (IS_ERR(attrs)) 6276 6246 return PTR_ERR(attrs); 6277 6247 ··· 6284 6254 return PTR_ERR(hist_data); 6285 6255 } 6286 6256 6287 - trigger_ops = cmd_ops->get_trigger_ops(cmd, trigger); 6288 - 6289 - trigger_data = kzalloc(sizeof(*trigger_data), GFP_KERNEL); 6257 + trigger_data = event_trigger_alloc(cmd_ops, cmd, param, hist_data); 6290 6258 if (!trigger_data) { 6291 6259 ret = -ENOMEM; 6292 6260 goto out_free; 6293 6261 } 6294 6262 6295 - trigger_data->count = -1; 6296 - trigger_data->ops = trigger_ops; 6297 - trigger_data->cmd_ops = cmd_ops; 6298 - 6299 - INIT_LIST_HEAD(&trigger_data->list); 6300 - RCU_INIT_POINTER(trigger_data->filter, NULL); 6301 - 6302 - trigger_data->private_data = hist_data; 6303 - 6304 - /* if param is non-empty, it's supposed to be a filter */ 6305 - if (param && cmd_ops->set_filter) { 6306 - ret = cmd_ops->set_filter(param, trigger_data, file); 6307 - if (ret < 0) 6308 - goto out_free; 6309 - } 6263 + ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data); 6264 + if (ret < 0) 6265 + goto out_free; 6310 6266 6311 6267 if (remove) { 6312 6268 if (!have_hist_trigger_match(trigger_data, file)) ··· 6303 6287 goto out_free; 6304 6288 } 6305 6289 6306 - cmd_ops->unreg(glob+1, trigger_data, file); 6290 + event_trigger_unregister(cmd_ops, file, glob+1, trigger_data); 6307 6291 se_name = trace_event_name(file->event_call); 6308 6292 se = find_synth_event(se_name); 6309 6293 if (se) ··· 6312 6296 goto out_free; 6313 6297 } 6314 6298 6315 - ret = cmd_ops->reg(glob, trigger_data, file); 6316 - /* 6317 - * The above returns on success the # of triggers registered, 6318 - * but if it didn't register any it returns zero. Consider no 6319 - * triggers registered a failure too. 6320 - */ 6321 - if (!ret) { 6322 - if (!(attrs->pause || attrs->cont || attrs->clear)) 6323 - ret = -ENOENT; 6299 + if (existing_hist_update_only(glob, trigger_data, file)) 6324 6300 goto out_free; 6325 - } else if (ret < 0) 6301 + 6302 + ret = event_trigger_register(cmd_ops, file, glob, trigger_data); 6303 + if (ret < 0) 6326 6304 goto out_free; 6327 6305 6328 6306 if (get_named_trigger_data(trigger_data)) ··· 6341 6331 se = find_synth_event(se_name); 6342 6332 if (se) 6343 6333 se->ref++; 6344 - /* Just return zero, not the number of registered triggers */ 6345 - ret = 0; 6346 6334 out: 6347 6335 if (ret == 0) 6348 6336 hist_err_clear(); 6349 6337 6350 6338 return ret; 6351 6339 out_unreg: 6352 - cmd_ops->unreg(glob+1, trigger_data, file); 6340 + event_trigger_unregister(cmd_ops, file, glob+1, trigger_data); 6353 6341 out_free: 6354 - if (cmd_ops->set_filter) 6355 - cmd_ops->set_filter(NULL, trigger_data, NULL); 6342 + event_trigger_reset_filter(cmd_ops, trigger_data); 6356 6343 6357 6344 remove_hist_vars(hist_data); 6358 6345 ··· 6470 6463 update_cond_flag(file); 6471 6464 trace_event_trigger_enable_disable(file, 0); 6472 6465 if (test->ops->free) 6473 - test->ops->free(test->ops, test); 6466 + test->ops->free(test); 6474 6467 } 6475 6468 } 6476 6469 }
+110 -218
kernel/trace/trace_events_trigger.c
··· 188 188 } 189 189 190 190 data = list_entry(v, struct event_trigger_data, list); 191 - data->ops->print(m, data->ops, data); 191 + data->ops->print(m, data); 192 192 193 193 return 0; 194 194 } ··· 432 432 433 433 /** 434 434 * event_trigger_init - Generic event_trigger_ops @init implementation 435 - * @ops: The trigger ops associated with the trigger 436 435 * @data: Trigger-specific data 437 436 * 438 437 * Common implementation of event trigger initialization. ··· 441 442 * 442 443 * Return: 0 on success, errno otherwise 443 444 */ 444 - int event_trigger_init(struct event_trigger_ops *ops, 445 - struct event_trigger_data *data) 445 + int event_trigger_init(struct event_trigger_data *data) 446 446 { 447 447 data->ref++; 448 448 return 0; ··· 449 451 450 452 /** 451 453 * event_trigger_free - Generic event_trigger_ops @free implementation 452 - * @ops: The trigger ops associated with the trigger 453 454 * @data: Trigger-specific data 454 455 * 455 456 * Common implementation of event trigger de-initialization. ··· 457 460 * implementations. 458 461 */ 459 462 static void 460 - event_trigger_free(struct event_trigger_ops *ops, 461 - struct event_trigger_data *data) 463 + event_trigger_free(struct event_trigger_data *data) 462 464 { 463 465 if (WARN_ON_ONCE(data->ref <= 0)) 464 466 return; ··· 511 515 trace_event_trigger_enable_disable(file, 0); 512 516 list_del_rcu(&data->list); 513 517 if (data->ops->free) 514 - data->ops->free(data->ops, data); 518 + data->ops->free(data); 515 519 } 516 520 } 517 521 } ··· 577 581 } 578 582 579 583 if (data->ops->init) { 580 - ret = data->ops->init(data->ops, data); 584 + ret = data->ops->init(data); 581 585 if (ret < 0) 582 586 goto out; 583 587 } 584 588 585 589 list_add_rcu(&data->list, &file->triggers); 586 - ret++; 587 590 588 591 update_cond_flag(file); 589 - if (trace_event_trigger_enable_disable(file, 1) < 0) { 592 + ret = trace_event_trigger_enable_disable(file, 1); 593 + if (ret < 0) { 590 594 list_del_rcu(&data->list); 591 595 update_cond_flag(file); 592 - ret--; 593 596 } 594 597 out: 595 598 return ret; ··· 609 614 struct event_trigger_data *test, 610 615 struct trace_event_file *file) 611 616 { 612 - struct event_trigger_data *data; 613 - bool unregistered = false; 617 + struct event_trigger_data *data = NULL, *iter; 614 618 615 619 lockdep_assert_held(&event_mutex); 616 620 617 - list_for_each_entry(data, &file->triggers, list) { 618 - if (data->cmd_ops->trigger_type == test->cmd_ops->trigger_type) { 619 - unregistered = true; 621 + list_for_each_entry(iter, &file->triggers, list) { 622 + if (iter->cmd_ops->trigger_type == test->cmd_ops->trigger_type) { 623 + data = iter; 620 624 list_del_rcu(&data->list); 621 625 trace_event_trigger_enable_disable(file, 0); 622 626 update_cond_flag(file); ··· 623 629 } 624 630 } 625 631 626 - if (unregistered && data->ops->free) 627 - data->ops->free(data->ops, data); 632 + if (data && data->ops->free) 633 + data->ops->free(data); 628 634 } 629 635 630 636 /* ··· 738 744 739 745 /** 740 746 * event_trigger_separate_filter - separate an event trigger from a filter 741 - * @param: The param string containing trigger and possibly filter 742 - * @trigger: outparam, will be filled with a pointer to the trigger 747 + * @param_and_filter: String containing trigger and possibly filter 748 + * @param: outparam, will be filled with a pointer to the trigger 743 749 * @filter: outparam, will be filled with a pointer to the filter 744 750 * @param_required: Specifies whether or not the param string is required 745 751 * 746 752 * Given a param string of the form '[trigger] [if filter]', this 747 753 * function separates the filter from the trigger and returns the 748 - * trigger in *trigger and the filter in *filter. Either the *trigger 749 - * or the *filter may be set to NULL by this function - if not set to 754 + * trigger in @param and the filter in @filter. Either the @param 755 + * or the @filter may be set to NULL by this function - if not set to 750 756 * NULL, they will contain strings corresponding to the trigger and 751 757 * filter. 752 758 * ··· 921 927 * @cmd_ops: The event_command operations for the trigger 922 928 * @file: The event file for the trigger's event 923 929 * @glob: The trigger command string, with optional remove(!) operator 924 - * @cmd: The cmd string 925 - * @param: The param string 926 930 * @trigger_data: The trigger_data for the trigger 927 - * @n_registered: optional outparam, the number of triggers registered 928 931 * 929 932 * Register an event trigger. The @cmd_ops are used to call the 930 - * cmd_ops->reg() function which actually does the registration. The 931 - * cmd_ops->reg() function returns the number of triggers registered, 932 - * which is assigned to n_registered, if n_registered is non-NULL. 933 + * cmd_ops->reg() function which actually does the registration. 933 934 * 934 935 * Return: 0 on success, errno otherwise 935 936 */ 936 937 int event_trigger_register(struct event_command *cmd_ops, 937 938 struct trace_event_file *file, 938 939 char *glob, 939 - char *cmd, 940 - char *param, 941 - struct event_trigger_data *trigger_data, 942 - int *n_registered) 940 + struct event_trigger_data *trigger_data) 943 941 { 944 - int ret; 942 + return cmd_ops->reg(glob, trigger_data, file); 943 + } 945 944 946 - if (n_registered) 947 - *n_registered = 0; 948 - 949 - ret = cmd_ops->reg(glob, trigger_data, file); 950 - /* 951 - * The above returns on success the # of functions enabled, 952 - * but if it didn't find any functions it returns zero. 953 - * Consider no functions a failure too. 954 - */ 955 - if (!ret) { 956 - cmd_ops->unreg(glob, trigger_data, file); 957 - ret = -ENOENT; 958 - } else if (ret > 0) { 959 - if (n_registered) 960 - *n_registered = ret; 961 - /* Just return zero, not the number of enabled functions */ 962 - ret = 0; 963 - } 964 - 965 - return ret; 945 + /** 946 + * event_trigger_unregister - unregister an event trigger 947 + * @cmd_ops: The event_command operations for the trigger 948 + * @file: The event file for the trigger's event 949 + * @glob: The trigger command string, with optional remove(!) operator 950 + * @trigger_data: The trigger_data for the trigger 951 + * 952 + * Unregister an event trigger. The @cmd_ops are used to call the 953 + * cmd_ops->unreg() function which actually does the unregistration. 954 + */ 955 + void event_trigger_unregister(struct event_command *cmd_ops, 956 + struct trace_event_file *file, 957 + char *glob, 958 + struct event_trigger_data *trigger_data) 959 + { 960 + cmd_ops->unreg(glob, trigger_data, file); 966 961 } 967 962 968 963 /* ··· 964 981 * @file: The trace_event_file associated with the event 965 982 * @glob: The raw string used to register the trigger 966 983 * @cmd: The cmd portion of the string used to register the trigger 967 - * @param: The params portion of the string used to register the trigger 984 + * @param_and_filter: The param and filter portion of the string used to register the trigger 968 985 * 969 986 * Common implementation for event command parsing and trigger 970 987 * instantiation. ··· 977 994 static int 978 995 event_trigger_parse(struct event_command *cmd_ops, 979 996 struct trace_event_file *file, 980 - char *glob, char *cmd, char *param) 997 + char *glob, char *cmd, char *param_and_filter) 981 998 { 982 999 struct event_trigger_data *trigger_data; 983 - struct event_trigger_ops *trigger_ops; 984 - char *trigger = NULL; 985 - char *number; 1000 + char *param, *filter; 1001 + bool remove; 986 1002 int ret; 987 1003 988 - /* separate the trigger from the filter (t:n [if filter]) */ 989 - if (param && isdigit(param[0])) { 990 - trigger = strsep(&param, " \t"); 991 - if (param) { 992 - param = skip_spaces(param); 993 - if (!*param) 994 - param = NULL; 995 - } 996 - } 1004 + remove = event_trigger_check_remove(glob); 997 1005 998 - trigger_ops = cmd_ops->get_trigger_ops(cmd, trigger); 1006 + ret = event_trigger_separate_filter(param_and_filter, &param, &filter, false); 1007 + if (ret) 1008 + return ret; 999 1009 1000 1010 ret = -ENOMEM; 1001 - trigger_data = kzalloc(sizeof(*trigger_data), GFP_KERNEL); 1011 + trigger_data = event_trigger_alloc(cmd_ops, cmd, param, file); 1002 1012 if (!trigger_data) 1003 1013 goto out; 1004 1014 1005 - trigger_data->count = -1; 1006 - trigger_data->ops = trigger_ops; 1007 - trigger_data->cmd_ops = cmd_ops; 1008 - trigger_data->private_data = file; 1009 - INIT_LIST_HEAD(&trigger_data->list); 1010 - INIT_LIST_HEAD(&trigger_data->named_list); 1011 - 1012 - if (glob[0] == '!') { 1013 - cmd_ops->unreg(glob+1, trigger_data, file); 1015 + if (remove) { 1016 + event_trigger_unregister(cmd_ops, file, glob+1, trigger_data); 1014 1017 kfree(trigger_data); 1015 1018 ret = 0; 1016 1019 goto out; 1017 1020 } 1018 1021 1019 - if (trigger) { 1020 - number = strsep(&trigger, ":"); 1022 + ret = event_trigger_parse_num(param, trigger_data); 1023 + if (ret) 1024 + goto out_free; 1021 1025 1022 - ret = -EINVAL; 1023 - if (!strlen(number)) 1024 - goto out_free; 1025 - 1026 - /* 1027 - * We use the callback data field (which is a pointer) 1028 - * as our counter. 1029 - */ 1030 - ret = kstrtoul(number, 0, &trigger_data->count); 1031 - if (ret) 1032 - goto out_free; 1033 - } 1034 - 1035 - if (!param) /* if param is non-empty, it's supposed to be a filter */ 1036 - goto out_reg; 1037 - 1038 - if (!cmd_ops->set_filter) 1039 - goto out_reg; 1040 - 1041 - ret = cmd_ops->set_filter(param, trigger_data, file); 1026 + ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data); 1042 1027 if (ret < 0) 1043 1028 goto out_free; 1044 1029 1045 - out_reg: 1046 1030 /* Up the trigger_data count to make sure reg doesn't free it on failure */ 1047 - event_trigger_init(trigger_ops, trigger_data); 1048 - ret = cmd_ops->reg(glob, trigger_data, file); 1049 - /* 1050 - * The above returns on success the # of functions enabled, 1051 - * but if it didn't find any functions it returns zero. 1052 - * Consider no functions a failure too. 1053 - */ 1054 - if (!ret) { 1055 - cmd_ops->unreg(glob, trigger_data, file); 1056 - ret = -ENOENT; 1057 - } else if (ret > 0) 1058 - ret = 0; 1031 + event_trigger_init(trigger_data); 1032 + 1033 + ret = event_trigger_register(cmd_ops, file, glob, trigger_data); 1034 + if (ret) 1035 + goto out_free; 1059 1036 1060 1037 /* Down the counter of trigger_data or free it if not used anymore */ 1061 - event_trigger_free(trigger_ops, trigger_data); 1038 + event_trigger_free(trigger_data); 1062 1039 out: 1063 1040 return ret; 1064 1041 1065 1042 out_free: 1066 - if (cmd_ops->set_filter) 1067 - cmd_ops->set_filter(NULL, trigger_data, NULL); 1043 + event_trigger_reset_filter(cmd_ops, trigger_data); 1068 1044 kfree(trigger_data); 1069 1045 goto out; 1070 1046 } ··· 1343 1401 } 1344 1402 1345 1403 static int 1346 - traceon_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, 1347 - struct event_trigger_data *data) 1404 + traceon_trigger_print(struct seq_file *m, struct event_trigger_data *data) 1348 1405 { 1349 1406 return event_trigger_print("traceon", m, (void *)data->count, 1350 1407 data->filter_str); 1351 1408 } 1352 1409 1353 1410 static int 1354 - traceoff_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, 1355 - struct event_trigger_data *data) 1411 + traceoff_trigger_print(struct seq_file *m, struct event_trigger_data *data) 1356 1412 { 1357 1413 return event_trigger_print("traceoff", m, (void *)data->count, 1358 1414 data->filter_str); ··· 1461 1521 } 1462 1522 1463 1523 static int 1464 - snapshot_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, 1465 - struct event_trigger_data *data) 1524 + snapshot_trigger_print(struct seq_file *m, struct event_trigger_data *data) 1466 1525 { 1467 1526 return event_trigger_print("snapshot", m, (void *)data->count, 1468 1527 data->filter_str); ··· 1556 1617 } 1557 1618 1558 1619 static int 1559 - stacktrace_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, 1560 - struct event_trigger_data *data) 1620 + stacktrace_trigger_print(struct seq_file *m, struct event_trigger_data *data) 1561 1621 { 1562 1622 return event_trigger_print("stacktrace", m, (void *)data->count, 1563 1623 data->filter_str); ··· 1646 1708 } 1647 1709 1648 1710 int event_enable_trigger_print(struct seq_file *m, 1649 - struct event_trigger_ops *ops, 1650 1711 struct event_trigger_data *data) 1651 1712 { 1652 1713 struct enable_trigger_data *enable_data = data->private_data; ··· 1670 1733 return 0; 1671 1734 } 1672 1735 1673 - void event_enable_trigger_free(struct event_trigger_ops *ops, 1674 - struct event_trigger_data *data) 1736 + void event_enable_trigger_free(struct event_trigger_data *data) 1675 1737 { 1676 1738 struct enable_trigger_data *enable_data = data->private_data; 1677 1739 ··· 1717 1781 1718 1782 int event_enable_trigger_parse(struct event_command *cmd_ops, 1719 1783 struct trace_event_file *file, 1720 - char *glob, char *cmd, char *param) 1784 + char *glob, char *cmd, char *param_and_filter) 1721 1785 { 1722 1786 struct trace_event_file *event_enable_file; 1723 1787 struct enable_trigger_data *enable_data; 1724 1788 struct event_trigger_data *trigger_data; 1725 - struct event_trigger_ops *trigger_ops; 1726 1789 struct trace_array *tr = file->tr; 1790 + char *param, *filter; 1791 + bool enable, remove; 1727 1792 const char *system; 1728 1793 const char *event; 1729 1794 bool hist = false; 1730 - char *trigger; 1731 - char *number; 1732 - bool enable; 1733 1795 int ret; 1734 1796 1797 + remove = event_trigger_check_remove(glob); 1798 + 1799 + if (event_trigger_empty_param(param_and_filter)) 1800 + return -EINVAL; 1801 + 1802 + ret = event_trigger_separate_filter(param_and_filter, &param, &filter, true); 1803 + if (ret) 1804 + return ret; 1805 + 1806 + system = strsep(&param, ":"); 1735 1807 if (!param) 1736 1808 return -EINVAL; 1737 1809 1738 - /* separate the trigger from the filter (s:e:n [if filter]) */ 1739 - trigger = strsep(&param, " \t"); 1740 - if (!trigger) 1741 - return -EINVAL; 1742 - if (param) { 1743 - param = skip_spaces(param); 1744 - if (!*param) 1745 - param = NULL; 1746 - } 1747 - 1748 - system = strsep(&trigger, ":"); 1749 - if (!trigger) 1750 - return -EINVAL; 1751 - 1752 - event = strsep(&trigger, ":"); 1810 + event = strsep(&param, ":"); 1753 1811 1754 1812 ret = -EINVAL; 1755 1813 event_enable_file = find_event_file(tr, system, event); ··· 1759 1829 #else 1760 1830 enable = strcmp(cmd, ENABLE_EVENT_STR) == 0; 1761 1831 #endif 1762 - trigger_ops = cmd_ops->get_trigger_ops(cmd, trigger); 1763 - 1764 1832 ret = -ENOMEM; 1765 - trigger_data = kzalloc(sizeof(*trigger_data), GFP_KERNEL); 1766 - if (!trigger_data) 1767 - goto out; 1768 1833 1769 1834 enable_data = kzalloc(sizeof(*enable_data), GFP_KERNEL); 1770 - if (!enable_data) { 1771 - kfree(trigger_data); 1835 + if (!enable_data) 1772 1836 goto out; 1773 - } 1774 - 1775 - trigger_data->count = -1; 1776 - trigger_data->ops = trigger_ops; 1777 - trigger_data->cmd_ops = cmd_ops; 1778 - INIT_LIST_HEAD(&trigger_data->list); 1779 - RCU_INIT_POINTER(trigger_data->filter, NULL); 1780 1837 1781 1838 enable_data->hist = hist; 1782 1839 enable_data->enable = enable; 1783 1840 enable_data->file = event_enable_file; 1784 - trigger_data->private_data = enable_data; 1785 1841 1786 - if (glob[0] == '!') { 1787 - cmd_ops->unreg(glob+1, trigger_data, file); 1842 + trigger_data = event_trigger_alloc(cmd_ops, cmd, param, enable_data); 1843 + if (!trigger_data) { 1844 + kfree(enable_data); 1845 + goto out; 1846 + } 1847 + 1848 + if (remove) { 1849 + event_trigger_unregister(cmd_ops, file, glob+1, trigger_data); 1788 1850 kfree(trigger_data); 1789 1851 kfree(enable_data); 1790 1852 ret = 0; ··· 1784 1862 } 1785 1863 1786 1864 /* Up the trigger_data count to make sure nothing frees it on failure */ 1787 - event_trigger_init(trigger_ops, trigger_data); 1865 + event_trigger_init(trigger_data); 1788 1866 1789 - if (trigger) { 1790 - number = strsep(&trigger, ":"); 1867 + ret = event_trigger_parse_num(param, trigger_data); 1868 + if (ret) 1869 + goto out_free; 1791 1870 1792 - ret = -EINVAL; 1793 - if (!strlen(number)) 1794 - goto out_free; 1795 - 1796 - /* 1797 - * We use the callback data field (which is a pointer) 1798 - * as our counter. 1799 - */ 1800 - ret = kstrtoul(number, 0, &trigger_data->count); 1801 - if (ret) 1802 - goto out_free; 1803 - } 1804 - 1805 - if (!param) /* if param is non-empty, it's supposed to be a filter */ 1806 - goto out_reg; 1807 - 1808 - if (!cmd_ops->set_filter) 1809 - goto out_reg; 1810 - 1811 - ret = cmd_ops->set_filter(param, trigger_data, file); 1871 + ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data); 1812 1872 if (ret < 0) 1813 1873 goto out_free; 1814 1874 1815 - out_reg: 1816 1875 /* Don't let event modules unload while probe registered */ 1817 1876 ret = trace_event_try_get_ref(event_enable_file->event_call); 1818 1877 if (!ret) { ··· 1804 1901 ret = trace_event_enable_disable(event_enable_file, 1, 1); 1805 1902 if (ret < 0) 1806 1903 goto out_put; 1807 - ret = cmd_ops->reg(glob, trigger_data, file); 1808 - /* 1809 - * The above returns on success the # of functions enabled, 1810 - * but if it didn't find any functions it returns zero. 1811 - * Consider no functions a failure too. 1812 - */ 1813 - if (!ret) { 1814 - ret = -ENOENT; 1904 + 1905 + ret = event_trigger_register(cmd_ops, file, glob, trigger_data); 1906 + if (ret) 1815 1907 goto out_disable; 1816 - } else if (ret < 0) 1817 - goto out_disable; 1818 - /* Just return zero, not the number of enabled functions */ 1819 - ret = 0; 1820 - event_trigger_free(trigger_ops, trigger_data); 1908 + 1909 + event_trigger_free(trigger_data); 1821 1910 out: 1822 1911 return ret; 1823 - 1824 1912 out_disable: 1825 1913 trace_event_enable_disable(event_enable_file, 0, 1); 1826 1914 out_put: 1827 1915 trace_event_put_ref(event_enable_file->event_call); 1828 1916 out_free: 1829 - if (cmd_ops->set_filter) 1830 - cmd_ops->set_filter(NULL, trigger_data, NULL); 1831 - event_trigger_free(trigger_ops, trigger_data); 1917 + event_trigger_reset_filter(cmd_ops, trigger_data); 1918 + event_trigger_free(trigger_data); 1832 1919 kfree(enable_data); 1920 + 1833 1921 goto out; 1834 1922 } 1835 1923 ··· 1847 1953 } 1848 1954 1849 1955 if (data->ops->init) { 1850 - ret = data->ops->init(data->ops, data); 1956 + ret = data->ops->init(data); 1851 1957 if (ret < 0) 1852 1958 goto out; 1853 1959 } 1854 1960 1855 1961 list_add_rcu(&data->list, &file->triggers); 1856 - ret++; 1857 1962 1858 1963 update_cond_flag(file); 1859 - if (trace_event_trigger_enable_disable(file, 1) < 0) { 1964 + ret = trace_event_trigger_enable_disable(file, 1); 1965 + if (ret < 0) { 1860 1966 list_del_rcu(&data->list); 1861 1967 update_cond_flag(file); 1862 - ret--; 1863 1968 } 1864 1969 out: 1865 1970 return ret; ··· 1869 1976 struct trace_event_file *file) 1870 1977 { 1871 1978 struct enable_trigger_data *test_enable_data = test->private_data; 1979 + struct event_trigger_data *data = NULL, *iter; 1872 1980 struct enable_trigger_data *enable_data; 1873 - struct event_trigger_data *data; 1874 - bool unregistered = false; 1875 1981 1876 1982 lockdep_assert_held(&event_mutex); 1877 1983 1878 - list_for_each_entry(data, &file->triggers, list) { 1879 - enable_data = data->private_data; 1984 + list_for_each_entry(iter, &file->triggers, list) { 1985 + enable_data = iter->private_data; 1880 1986 if (enable_data && 1881 - (data->cmd_ops->trigger_type == 1987 + (iter->cmd_ops->trigger_type == 1882 1988 test->cmd_ops->trigger_type) && 1883 1989 (enable_data->file == test_enable_data->file)) { 1884 - unregistered = true; 1990 + data = iter; 1885 1991 list_del_rcu(&data->list); 1886 1992 trace_event_trigger_enable_disable(file, 0); 1887 1993 update_cond_flag(file); ··· 1888 1996 } 1889 1997 } 1890 1998 1891 - if (unregistered && data->ops->free) 1892 - data->ops->free(data->ops, data); 1999 + if (data && data->ops->free) 2000 + data->ops->free(data); 1893 2001 } 1894 2002 1895 2003 static struct event_trigger_ops *
+4 -11
kernel/trace/trace_kprobe.c
··· 1907 1907 static __init int init_kprobe_trace(void) 1908 1908 { 1909 1909 int ret; 1910 - struct dentry *entry; 1911 1910 1912 1911 ret = tracing_init_dentry(); 1913 1912 if (ret) 1914 1913 return 0; 1915 1914 1916 - entry = tracefs_create_file("kprobe_events", TRACE_MODE_WRITE, 1917 - NULL, NULL, &kprobe_events_ops); 1918 - 1919 1915 /* Event list interface */ 1920 - if (!entry) 1921 - pr_warn("Could not create tracefs 'kprobe_events' entry\n"); 1916 + trace_create_file("kprobe_events", TRACE_MODE_WRITE, 1917 + NULL, NULL, &kprobe_events_ops); 1922 1918 1923 1919 /* Profile interface */ 1924 - entry = tracefs_create_file("kprobe_profile", TRACE_MODE_READ, 1925 - NULL, NULL, &kprobe_profile_ops); 1926 - 1927 - if (!entry) 1928 - pr_warn("Could not create tracefs 'kprobe_profile' entry\n"); 1920 + trace_create_file("kprobe_profile", TRACE_MODE_READ, 1921 + NULL, NULL, &kprobe_profile_ops); 1929 1922 1930 1923 setup_boot_kprobe_events(); 1931 1924
+19 -3
kernel/trace/trace_osnoise.c
··· 1578 1578 1579 1579 trace_timerlat_sample(&s); 1580 1580 1581 - notify_new_max_latency(diff); 1581 + if (osnoise_data.stop_tracing) { 1582 + if (time_to_us(diff) >= osnoise_data.stop_tracing) { 1582 1583 1583 - if (osnoise_data.stop_tracing) 1584 - if (time_to_us(diff) >= osnoise_data.stop_tracing) 1584 + /* 1585 + * At this point, if stop_tracing is set and <= print_stack, 1586 + * print_stack is set and would be printed in the thread handler. 1587 + * 1588 + * Thus, print the stack trace as it is helpful to define the 1589 + * root cause of an IRQ latency. 1590 + */ 1591 + if (osnoise_data.stop_tracing <= osnoise_data.print_stack) { 1592 + timerlat_save_stack(0); 1593 + timerlat_dump_stack(time_to_us(diff)); 1594 + } 1595 + 1585 1596 osnoise_stop_tracing(); 1597 + notify_new_max_latency(diff); 1598 + 1599 + return HRTIMER_NORESTART; 1600 + } 1601 + } 1586 1602 1587 1603 wake_up_process(tlat->kthread); 1588 1604
+12 -13
kernel/trace/trace_output.c
··· 692 692 693 693 static int trace_search_list(struct list_head **list) 694 694 { 695 - struct trace_event *e; 695 + struct trace_event *e = NULL, *iter; 696 696 int next = __TRACE_LAST_TYPE; 697 697 698 698 if (list_empty(&ftrace_event_list)) { ··· 704 704 * We used up all possible max events, 705 705 * lets see if somebody freed one. 706 706 */ 707 - list_for_each_entry(e, &ftrace_event_list, list) { 708 - if (e->type != next) 707 + list_for_each_entry(iter, &ftrace_event_list, list) { 708 + if (iter->type != next) { 709 + e = iter; 709 710 break; 711 + } 710 712 next++; 711 713 } 712 714 ··· 716 714 if (next > TRACE_EVENT_TYPE_MAX) 717 715 return 0; 718 716 719 - *list = &e->list; 717 + if (e) 718 + *list = &e->list; 719 + else 720 + *list = &ftrace_event_list; 720 721 return next; 721 722 } 722 723 ··· 783 778 784 779 list_add_tail(&event->list, list); 785 780 786 - } else if (event->type > __TRACE_LAST_TYPE) { 787 - printk(KERN_WARNING "Need to add type to trace.h\n"); 788 - WARN_ON(1); 781 + } else if (WARN(event->type > __TRACE_LAST_TYPE, 782 + "Need to add type to trace.h")) { 789 783 goto out; 790 784 } else { 791 785 /* Is this event already used */ ··· 1575 1571 1576 1572 for (i = 0; events[i]; i++) { 1577 1573 event = events[i]; 1578 - 1579 1574 ret = register_trace_event(event); 1580 - if (!ret) { 1581 - printk(KERN_WARNING "event %d failed to register\n", 1582 - event->type); 1583 - WARN_ON_ONCE(1); 1584 - } 1575 + WARN_ONCE(!ret, "event %d failed to register", event->type); 1585 1576 } 1586 1577 1587 1578 return 0;
+2 -5
kernel/trace/trace_recursion_record.c
··· 224 224 225 225 __init static int create_recursed_functions(void) 226 226 { 227 - struct dentry *dentry; 228 227 229 - dentry = trace_create_file("recursed_functions", TRACE_MODE_WRITE, 230 - NULL, NULL, &recursed_functions_fops); 231 - if (!dentry) 232 - pr_warn("WARNING: Failed to create recursed_functions\n"); 228 + trace_create_file("recursed_functions", TRACE_MODE_WRITE, 229 + NULL, NULL, &recursed_functions_fops); 233 230 return 0; 234 231 } 235 232
+3
kernel/trace/trace_selftest.c
··· 895 895 ret = -1; 896 896 goto out; 897 897 } 898 + 899 + /* Enable tracing on all functions again */ 900 + ftrace_set_global_filter(NULL, 0, 1); 898 901 #endif 899 902 900 903 /* Don't test dynamic tracing, the function tracer already did */
+11 -24
kernel/trace/trace_syscalls.c
··· 154 154 goto end; 155 155 156 156 /* parameter types */ 157 - if (tr->trace_flags & TRACE_ITER_VERBOSE) 157 + if (tr && tr->trace_flags & TRACE_ITER_VERBOSE) 158 158 trace_seq_printf(s, "%s ", entry->types[i]); 159 159 160 160 /* parameter values */ ··· 296 296 struct trace_event_file *trace_file; 297 297 struct syscall_trace_enter *entry; 298 298 struct syscall_metadata *sys_data; 299 - struct ring_buffer_event *event; 300 - struct trace_buffer *buffer; 301 - unsigned int trace_ctx; 299 + struct trace_event_buffer fbuffer; 302 300 unsigned long args[6]; 303 301 int syscall_nr; 304 302 int size; ··· 319 321 320 322 size = sizeof(*entry) + sizeof(unsigned long) * sys_data->nb_args; 321 323 322 - trace_ctx = tracing_gen_ctx(); 323 - 324 - event = trace_event_buffer_lock_reserve(&buffer, trace_file, 325 - sys_data->enter_event->event.type, size, trace_ctx); 326 - if (!event) 324 + entry = trace_event_buffer_reserve(&fbuffer, trace_file, size); 325 + if (!entry) 327 326 return; 328 327 329 - entry = ring_buffer_event_data(event); 328 + entry = ring_buffer_event_data(fbuffer.event); 330 329 entry->nr = syscall_nr; 331 330 syscall_get_arguments(current, regs, args); 332 331 memcpy(entry->args, args, sizeof(unsigned long) * sys_data->nb_args); 333 332 334 - event_trigger_unlock_commit(trace_file, buffer, event, entry, 335 - trace_ctx); 333 + trace_event_buffer_commit(&fbuffer); 336 334 } 337 335 338 336 static void ftrace_syscall_exit(void *data, struct pt_regs *regs, long ret) ··· 337 343 struct trace_event_file *trace_file; 338 344 struct syscall_trace_exit *entry; 339 345 struct syscall_metadata *sys_data; 340 - struct ring_buffer_event *event; 341 - struct trace_buffer *buffer; 342 - unsigned int trace_ctx; 346 + struct trace_event_buffer fbuffer; 343 347 int syscall_nr; 344 348 345 349 syscall_nr = trace_get_syscall_nr(current, regs); ··· 356 364 if (!sys_data) 357 365 return; 358 366 359 - trace_ctx = tracing_gen_ctx(); 360 - 361 - event = trace_event_buffer_lock_reserve(&buffer, trace_file, 362 - sys_data->exit_event->event.type, sizeof(*entry), 363 - trace_ctx); 364 - if (!event) 367 + entry = trace_event_buffer_reserve(&fbuffer, trace_file, sizeof(*entry)); 368 + if (!entry) 365 369 return; 366 370 367 - entry = ring_buffer_event_data(event); 371 + entry = ring_buffer_event_data(fbuffer.event); 368 372 entry->nr = syscall_nr; 369 373 entry->ret = syscall_get_return_value(current, regs); 370 374 371 - event_trigger_unlock_commit(trace_file, buffer, event, entry, 372 - trace_ctx); 375 + trace_event_buffer_commit(&fbuffer); 373 376 } 374 377 375 378 static int reg_event_syscall_enter(struct trace_event_file *file,
+2 -1
kernel/trace/tracing_map.c
··· 1045 1045 /** 1046 1046 * tracing_map_sort_entries - Sort the current set of tracing_map_elts in a map 1047 1047 * @map: The tracing_map 1048 - * @sort_key: The sort key to use for sorting 1048 + * @sort_keys: The sort key to use for sorting 1049 + * @n_sort_keys: hitcount, always have at least one 1049 1050 * @sort_entries: outval: pointer to allocated and sorted array of entries 1050 1051 * 1051 1052 * tracing_map_sort_entries() sorts the current set of entries in the
+1
lib/.gitignore
··· 1 1 # SPDX-License-Identifier: GPL-2.0-only 2 2 /crc32table.h 3 3 /crc64table.h 4 + /default.bconf 4 5 /gen_crc32table 5 6 /gen_crc64table 6 7 /oid_registry_data.c
+9 -1
lib/Makefile
··· 281 281 $(eval CFLAGS_$(file) = -I $(srctree)/scripts/dtc/libfdt)) 282 282 lib-$(CONFIG_LIBFDT) += $(libfdt_files) 283 283 284 - lib-$(CONFIG_BOOT_CONFIG) += bootconfig.o 284 + obj-$(CONFIG_BOOT_CONFIG) += bootconfig.o 285 + obj-$(CONFIG_BOOT_CONFIG_EMBED) += bootconfig-data.o 286 + 287 + $(obj)/bootconfig-data.o: $(obj)/default.bconf 288 + 289 + targets += default.bconf 290 + filechk_defbconf = cat $(or $(real-prereqs), /dev/null) 291 + $(obj)/default.bconf: $(CONFIG_BOOT_CONFIG_EMBED_FILE) FORCE 292 + $(call filechk,defbconf) 285 293 286 294 obj-$(CONFIG_RBTREE_TEST) += rbtree_test.o 287 295 obj-$(CONFIG_INTERVAL_TREE_TEST) += interval_tree_test.o
+10
lib/bootconfig-data.S
··· 1 + /* SPDX-License-Identifier: GPL-2.0 */ 2 + /* 3 + * Embed default bootconfig in the kernel. 4 + */ 5 + .section .init.rodata, "aw" 6 + .global embedded_bootconfig_data 7 + embedded_bootconfig_data: 8 + .incbin "lib/default.bconf" 9 + .global embedded_bootconfig_data_end 10 + embedded_bootconfig_data_end:
+13
lib/bootconfig.c
··· 12 12 #include <linux/kernel.h> 13 13 #include <linux/memblock.h> 14 14 #include <linux/string.h> 15 + 16 + #ifdef CONFIG_BOOT_CONFIG_EMBED 17 + /* embedded_bootconfig_data is defined in bootconfig-data.S */ 18 + extern __visible const char embedded_bootconfig_data[]; 19 + extern __visible const char embedded_bootconfig_data_end[]; 20 + 21 + const char * __init xbc_get_embedded_bootconfig(size_t *size) 22 + { 23 + *size = embedded_bootconfig_data_end - embedded_bootconfig_data; 24 + return (*size) ? embedded_bootconfig_data : NULL; 25 + } 26 + #endif 27 + 15 28 #else /* !__KERNEL__ */ 16 29 /* 17 30 * NOTE: This is only for tools/bootconfig, because tools/bootconfig will