Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

net: add add indirect call wrapper in skb_release_head_state()

While stress testing UDP senders on a host with expensive indirect
calls, I found cpus processing TX completions where showing
a very high cost (20%) in sock_wfree() due to
CONFIG_MITIGATION_RETPOLINE=y.

Take care of TCP and UDP TX destructors and use INDIRECT_CALL_3() macro.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Tested-by: Jamal Hadi Salim <jhs@mojatatu.com>
Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>
Link: https://patch.msgid.link/20251014171907.3554413-3-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Eric Dumazet and committed by
Jakub Kicinski
5b2b7dec 56cef47c

+10 -1
+10 -1
net/core/skbuff.c
··· 1136 1136 skb_dst_drop(skb); 1137 1137 if (skb->destructor) { 1138 1138 DEBUG_NET_WARN_ON_ONCE(in_hardirq()); 1139 - skb->destructor(skb); 1139 + #ifdef CONFIG_INET 1140 + INDIRECT_CALL_3(skb->destructor, 1141 + tcp_wfree, __sock_wfree, sock_wfree, 1142 + skb); 1143 + #else 1144 + INDIRECT_CALL_1(skb->destructor, 1145 + sock_wfree, 1146 + skb); 1147 + 1148 + #endif 1140 1149 } 1141 1150 #if IS_ENABLED(CONFIG_NF_CONNTRACK) 1142 1151 nf_conntrack_put(skb_nfct(skb));