Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

vhost-net: reduce one userspace copy when building XDP buff

We used to do twice copy_from_iter() to copy virtio-net and packet
separately. This introduce overheads for userspace access hardening as
well as SMAP (for x86 it's stac/clac). So this patch tries to use one
copy_from_iter() to copy them once and move the virtio-net header
afterwards to reduce overheads.

Testpmd + vhost_net shows 10% improvement from 5.45Mpps to 6.0Mpps.

Signed-off-by: Jason Wang <jasowang@redhat.com>
Link: https://patch.msgid.link/20250701010352.74515-2-jasowang@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Jason Wang and committed by
Jakub Kicinski
97b2409f 4d313f2b

+6 -10
+6 -10
drivers/vhost/net.c
··· 690 690 if (unlikely(!buf)) 691 691 return -ENOMEM; 692 692 693 - copied = copy_from_iter(buf, sock_hlen, from); 694 - if (copied != sock_hlen) { 693 + copied = copy_from_iter(buf + pad - sock_hlen, len, from); 694 + if (copied != len) { 695 695 ret = -EFAULT; 696 696 goto err; 697 697 } 698 698 699 - gso = buf; 699 + gso = buf + pad - sock_hlen; 700 700 701 701 if (!sock_hlen) 702 702 memset(buf, 0, pad); ··· 715 715 } 716 716 } 717 717 718 - len -= sock_hlen; 719 - copied = copy_from_iter(buf + pad, len, from); 720 - if (copied != len) { 721 - ret = -EFAULT; 722 - goto err; 723 - } 718 + /* pad contains sock_hlen */ 719 + memcpy(buf, buf + pad - sock_hlen, sock_hlen); 724 720 725 721 xdp_init_buff(xdp, buflen, NULL); 726 - xdp_prepare_buff(xdp, buf, pad, len, true); 722 + xdp_prepare_buff(xdp, buf, pad, len - sock_hlen, true); 727 723 728 724 ++nvq->batched_xdp; 729 725