Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

bpf: Improve program stats run-time calculation

This patch improves the run-time calculation for program stats by
capturing the duration as soon as possible after the program returns.

Previously, the duration included u64_stats_t operations. While the
instrumentation overhead is part of the total time spent when stats are
enabled, distinguishing between the program's native execution time and
the time spent due to instrumentation is crucial for accurate
performance analysis.

By making this change, the patch facilitates more precise optimization
of BPF programs, enabling users to understand their performance in
environments without stats enabled.

I used a virtualized environment to measure the run-time over one minute
for a basic raw_tracepoint/sys_enter program, which just increments a
local counter. Although the virtualization introduced some performance
degradation that could affect the results, I observed approximately a
16% decrease in average run-time reported by stats with this change
(310 -> 260 nsec).

Signed-off-by: Jose Fernandez <josef@netflix.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20240402034010.25060-1-josef@netflix.com

authored by

Jose Fernandez and committed by
Daniel Borkmann
ce09cbdd c186ed12

+6 -3
+4 -2
include/linux/filter.h
··· 654 654 cant_migrate(); 655 655 if (static_branch_unlikely(&bpf_stats_enabled_key)) { 656 656 struct bpf_prog_stats *stats; 657 - u64 start = sched_clock(); 657 + u64 duration, start = sched_clock(); 658 658 unsigned long flags; 659 659 660 660 ret = dfunc(ctx, prog->insnsi, prog->bpf_func); 661 + 662 + duration = sched_clock() - start; 661 663 stats = this_cpu_ptr(prog->stats); 662 664 flags = u64_stats_update_begin_irqsave(&stats->syncp); 663 665 u64_stats_inc(&stats->cnt); 664 - u64_stats_add(&stats->nsecs, sched_clock() - start); 666 + u64_stats_add(&stats->nsecs, duration); 665 667 u64_stats_update_end_irqrestore(&stats->syncp, flags); 666 668 } else { 667 669 ret = dfunc(ctx, prog->insnsi, prog->bpf_func);
+2 -1
kernel/bpf/trampoline.c
··· 885 885 * Hence check that 'start' is valid. 886 886 */ 887 887 start > NO_START_TIME) { 888 + u64 duration = sched_clock() - start; 888 889 unsigned long flags; 889 890 890 891 stats = this_cpu_ptr(prog->stats); 891 892 flags = u64_stats_update_begin_irqsave(&stats->syncp); 892 893 u64_stats_inc(&stats->cnt); 893 - u64_stats_add(&stats->nsecs, sched_clock() - start); 894 + u64_stats_add(&stats->nsecs, duration); 894 895 u64_stats_update_end_irqrestore(&stats->syncp, flags); 895 896 } 896 897 }