Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

[PKT_SCHED]: improve hashing performance of cls_fw

Calculate hashtable size to fit into a page instead of a hardcoded
256 buckets hash table. Results in a 1024 buckets hashtable on
most systems.

Replace old naive extract-8-lsb-bits algorithm with a better
algorithm xor'ing 3 or 4 bit fields at the size of the hashtable
array index in order to improve distribution if the majority of
the lower bits are unused while keeping zero collision behaviour
for the most common use case.

Thanks to Wang Jian <lark@linux.net.cn> for bringing this issue
to attention and to Eran Mann <emann@mrv.com> for the initial
idea for this new algorithm.

Signed-off-by: Thomas Graf <tgraf@suug.ch>
Signed-off-by: David S. Miller <davem@davemloft.net>

authored by

Thomas Graf and committed by
David S. Miller
c5c13faf 0d3d077c

+27 -4
+27 -4
net/sched/cls_fw.c
··· 46 46 #include <net/act_api.h> 47 47 #include <net/pkt_cls.h> 48 48 49 + #define HTSIZE (PAGE_SIZE/sizeof(struct fw_filter *)) 50 + 49 51 struct fw_head 50 52 { 51 - struct fw_filter *ht[256]; 53 + struct fw_filter *ht[HTSIZE]; 52 54 }; 53 55 54 56 struct fw_filter ··· 71 69 72 70 static __inline__ int fw_hash(u32 handle) 73 71 { 74 - return handle&0xFF; 72 + if (HTSIZE == 4096) 73 + return ((handle >> 24) & 0xFFF) ^ 74 + ((handle >> 12) & 0xFFF) ^ 75 + (handle & 0xFFF); 76 + else if (HTSIZE == 2048) 77 + return ((handle >> 22) & 0x7FF) ^ 78 + ((handle >> 11) & 0x7FF) ^ 79 + (handle & 0x7FF); 80 + else if (HTSIZE == 1024) 81 + return ((handle >> 20) & 0x3FF) ^ 82 + ((handle >> 10) & 0x3FF) ^ 83 + (handle & 0x3FF); 84 + else if (HTSIZE == 512) 85 + return (handle >> 27) ^ 86 + ((handle >> 18) & 0x1FF) ^ 87 + ((handle >> 9) & 0x1FF) ^ 88 + (handle & 0x1FF); 89 + else if (HTSIZE == 256) { 90 + u8 *t = (u8 *) &handle; 91 + return t[0] ^ t[1] ^ t[2] ^ t[3]; 92 + } else 93 + return handle & (HTSIZE - 1); 75 94 } 76 95 77 96 static int fw_classify(struct sk_buff *skb, struct tcf_proto *tp, ··· 175 152 if (head == NULL) 176 153 return; 177 154 178 - for (h=0; h<256; h++) { 155 + for (h=0; h<HTSIZE; h++) { 179 156 while ((f=head->ht[h]) != NULL) { 180 157 head->ht[h] = f->next; 181 158 fw_delete_filter(tp, f); ··· 314 291 if (arg->stop) 315 292 return; 316 293 317 - for (h = 0; h < 256; h++) { 294 + for (h = 0; h < HTSIZE; h++) { 318 295 struct fw_filter *f; 319 296 320 297 for (f = head->ht[h]; f; f = f->next) {