Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

xfrm: Skip redundant replay recheck for the hardware offload path

The xfrm_replay_recheck() function was introduced to handle the issues
arising from asynchronous crypto algorithms.

The crypto offload path is now effectively synchronous, as it holds
the state lock throughout its operation. This eliminates the race
condition, making the recheck an unnecessary overhead. This patch
improves performance by skipping the redundant call when
crypto_done is true.

Additionally, the sequence number assignment is moved to an earlier
point in the function. This improves performance by reducing lock
contention and places the logic at a more appropriate point, as the
full sequence number (including the higher-order bits) can be
determined as soon as the packet is received.

Signed-off-by: Jianbo Liu <jianbol@nvidia.com>
Reviewed-by: Cosmin Ratiu <cratiu@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>

authored by

Jianbo Liu and committed by
Steffen Klassert
b427c0c3 10a11861

+8 -8
+8 -8
net/xfrm/xfrm_input.c
··· 546 546 nexthdr = x->type_offload->input_tail(x, skb); 547 547 } 548 548 549 - goto lock; 549 + goto process; 550 550 } 551 551 552 552 family = XFRM_SPI_SKB_CB(skb)->family; ··· 614 614 goto drop; 615 615 } 616 616 617 - lock: 617 + process: 618 + seq_hi = htonl(xfrm_replay_seqhi(x, seq)); 619 + 620 + XFRM_SKB_CB(skb)->seq.input.low = seq; 621 + XFRM_SKB_CB(skb)->seq.input.hi = seq_hi; 622 + 618 623 spin_lock(&x->lock); 619 624 620 625 if (unlikely(x->km.state != XFRM_STATE_VALID)) { ··· 651 646 goto drop_unlock; 652 647 } 653 648 654 - seq_hi = htonl(xfrm_replay_seqhi(x, seq)); 655 - 656 - XFRM_SKB_CB(skb)->seq.input.low = seq; 657 - XFRM_SKB_CB(skb)->seq.input.hi = seq_hi; 658 - 659 649 if (!crypto_done) { 660 650 spin_unlock(&x->lock); 661 651 dev_hold(skb->dev); ··· 676 676 /* only the first xfrm gets the encap type */ 677 677 encap_type = 0; 678 678 679 - if (xfrm_replay_recheck(x, skb, seq)) { 679 + if (!crypto_done && xfrm_replay_recheck(x, skb, seq)) { 680 680 XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATESEQERROR); 681 681 goto drop_unlock; 682 682 }