Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

vp_vdpa: don't allocate unused msix vectors

When there is a ctlq and it doesn't require interrupt
callbacks,the original method of calculating vectors
wastes hardware msi or msix resources as well as system
IRQ resources.

When conducting performance testing using testpmd in the
guest os, it was found that the performance was lower compared
to directly using vfio-pci to passthrough the device

In scenarios where the virtio device in the guest os does
not utilize interrupts, the vdpa driver still configures
the hardware's msix vector. Therefore, the hardware still
sends interrupts to the host os. Because of this unnecessary
action by the hardware, hardware performance decreases, and
it also affects the performance of the host os.

Before modification:(interrupt mode)
32: 0 0 0 0 PCI-MSI 32768-edge vp-vdpa[0000:00:02.0]-0
33: 0 0 0 0 PCI-MSI 32769-edge vp-vdpa[0000:00:02.0]-1
34: 0 0 0 0 PCI-MSI 32770-edge vp-vdpa[0000:00:02.0]-2
35: 0 0 0 0 PCI-MSI 32771-edge vp-vdpa[0000:00:02.0]-config

After modification:(interrupt mode)
32: 0 0 1 7 PCI-MSI 32768-edge vp-vdpa[0000:00:02.0]-0
33: 36 0 3 0 PCI-MSI 32769-edge vp-vdpa[0000:00:02.0]-1
34: 0 0 0 0 PCI-MSI 32770-edge vp-vdpa[0000:00:02.0]-config

Before modification:(virtio pmd mode for guest os)
32: 0 0 0 0 PCI-MSI 32768-edge vp-vdpa[0000:00:02.0]-0
33: 0 0 0 0 PCI-MSI 32769-edge vp-vdpa[0000:00:02.0]-1
34: 0 0 0 0 PCI-MSI 32770-edge vp-vdpa[0000:00:02.0]-2
35: 0 0 0 0 PCI-MSI 32771-edge vp-vdpa[0000:00:02.0]-config

After modification:(virtio pmd mode for guest os)
32: 0 0 0 0 PCI-MSI 32768-edge vp-vdpa[0000:00:02.0]-config

To verify the use of the virtio PMD mode in the guest operating
system, the following patch needs to be applied to QEMU:
https://lore.kernel.org/all/20240408073311.2049-1-yuxue.liu@jaguarmicro.com

Signed-off-by: Yuxue Liu <yuxue.liu@jaguarmicro.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Heng Qi <hengqi@linux.alibaba.com>
Message-Id: <20240410033020.1310-1-yuxue.liu@jaguarmicro.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

authored by

Yuxue Liu and committed by
Michael S. Tsirkin
4d685629 1fa74f24

+16 -6
+16 -6
drivers/vdpa/virtio_pci/vp_vdpa.c
··· 160 160 struct pci_dev *pdev = mdev->pci_dev; 161 161 int i, ret, irq; 162 162 int queues = vp_vdpa->queues; 163 - int vectors = queues + 1; 163 + int vectors = 1; 164 + int msix_vec = 0; 165 + 166 + for (i = 0; i < queues; i++) { 167 + if (vp_vdpa->vring[i].cb.callback) 168 + vectors++; 169 + } 164 170 165 171 ret = pci_alloc_irq_vectors(pdev, vectors, vectors, PCI_IRQ_MSIX); 166 172 if (ret != vectors) { ··· 179 173 vp_vdpa->vectors = vectors; 180 174 181 175 for (i = 0; i < queues; i++) { 176 + if (!vp_vdpa->vring[i].cb.callback) 177 + continue; 178 + 182 179 snprintf(vp_vdpa->vring[i].msix_name, VP_VDPA_NAME_SIZE, 183 180 "vp-vdpa[%s]-%d\n", pci_name(pdev), i); 184 - irq = pci_irq_vector(pdev, i); 181 + irq = pci_irq_vector(pdev, msix_vec); 185 182 ret = devm_request_irq(&pdev->dev, irq, 186 183 vp_vdpa_vq_handler, 187 184 0, vp_vdpa->vring[i].msix_name, ··· 194 185 "vp_vdpa: fail to request irq for vq %d\n", i); 195 186 goto err; 196 187 } 197 - vp_modern_queue_vector(mdev, i, i); 188 + vp_modern_queue_vector(mdev, i, msix_vec); 198 189 vp_vdpa->vring[i].irq = irq; 190 + msix_vec++; 199 191 } 200 192 201 193 snprintf(vp_vdpa->msix_name, VP_VDPA_NAME_SIZE, "vp-vdpa[%s]-config\n", 202 194 pci_name(pdev)); 203 - irq = pci_irq_vector(pdev, queues); 195 + irq = pci_irq_vector(pdev, msix_vec); 204 196 ret = devm_request_irq(&pdev->dev, irq, vp_vdpa_config_handler, 0, 205 197 vp_vdpa->msix_name, vp_vdpa); 206 198 if (ret) { 207 199 dev_err(&pdev->dev, 208 - "vp_vdpa: fail to request irq for vq %d\n", i); 200 + "vp_vdpa: fail to request irq for config: %d\n", ret); 209 201 goto err; 210 202 } 211 - vp_modern_config_vector(mdev, queues); 203 + vp_modern_config_vector(mdev, msix_vec); 212 204 vp_vdpa->config_irq = irq; 213 205 214 206 return 0;