Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

tcp/dccp: bypass empty buckets in inet_twsk_purge()

TCP ehash table is often sparsely populated.

inet_twsk_purge() spends too much time calling cond_resched().

This patch can reduce time spent in inet_twsk_purge() by 20x.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Link: https://lore.kernel.org/r/20240327191206.508114-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Eric Dumazet and committed by
Jakub Kicinski
50e2907e 385edf32

+7 -2
+7 -2
net/ipv4/inet_timewait_sock.c
··· 266 266 /* Remove all non full sockets (TIME_WAIT and NEW_SYN_RECV) for dead netns */ 267 267 void inet_twsk_purge(struct inet_hashinfo *hashinfo, int family) 268 268 { 269 + struct inet_ehash_bucket *head = &hashinfo->ehash[0]; 270 + unsigned int ehash_mask = hashinfo->ehash_mask; 269 271 struct hlist_nulls_node *node; 270 272 unsigned int slot; 271 273 struct sock *sk; 272 274 273 - for (slot = 0; slot <= hashinfo->ehash_mask; slot++) { 274 - struct inet_ehash_bucket *head = &hashinfo->ehash[slot]; 275 + for (slot = 0; slot <= ehash_mask; slot++, head++) { 276 + 277 + if (hlist_nulls_empty(&head->chain)) 278 + continue; 279 + 275 280 restart_rcu: 276 281 cond_resched(); 277 282 rcu_read_lock();