Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

net: Document netmem driver support

Document expectations from drivers looking to add support for device
memory tcp or other netmem based features.

Signed-off-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>
Reviewed-by: Shannon Nelson <shannon.nelson@amd.com>
Link: https://patch.msgid.link/20241217201206.2360389-1-almasrymina@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Mina Almasry and committed by
Jakub Kicinski
f6038d91 0c7469ee

+80
+1
Documentation/networking/index.rst
··· 86 86 netdevices 87 87 netfilter-sysctl 88 88 netif-msg 89 + netmem 89 90 nexthop-group-resilient 90 91 nf_conntrack-sysctl 91 92 nf_flowtable
+79
Documentation/networking/netmem.rst
··· 1 + .. SPDX-License-Identifier: GPL-2.0 2 + 3 + ================================== 4 + Netmem Support for Network Drivers 5 + ================================== 6 + 7 + This document outlines the requirements for network drivers to support netmem, 8 + an abstract memory type that enables features like device memory TCP. By 9 + supporting netmem, drivers can work with various underlying memory types 10 + with little to no modification. 11 + 12 + Benefits of Netmem : 13 + 14 + * Flexibility: Netmem can be backed by different memory types (e.g., struct 15 + page, DMA-buf), allowing drivers to support various use cases such as device 16 + memory TCP. 17 + * Future-proof: Drivers with netmem support are ready for upcoming 18 + features that rely on it. 19 + * Simplified Development: Drivers interact with a consistent API, 20 + regardless of the underlying memory implementation. 21 + 22 + Driver Requirements 23 + =================== 24 + 25 + 1. The driver must support page_pool. 26 + 27 + 2. The driver must support the tcp-data-split ethtool option. 28 + 29 + 3. The driver must use the page_pool netmem APIs for payload memory. The netmem 30 + APIs currently 1-to-1 correspond with page APIs. Conversion to netmem should 31 + be achievable by switching the page APIs to netmem APIs and tracking memory 32 + via netmem_refs in the driver rather than struct page * : 33 + 34 + - page_pool_alloc -> page_pool_alloc_netmem 35 + - page_pool_get_dma_addr -> page_pool_get_dma_addr_netmem 36 + - page_pool_put_page -> page_pool_put_netmem 37 + 38 + Not all page APIs have netmem equivalents at the moment. If your driver 39 + relies on a missing netmem API, feel free to add and propose to netdev@, or 40 + reach out to the maintainers and/or almasrymina@google.com for help adding 41 + the netmem API. 42 + 43 + 4. The driver must use the following PP_FLAGS: 44 + 45 + - PP_FLAG_DMA_MAP: netmem is not dma-mappable by the driver. The driver 46 + must delegate the dma mapping to the page_pool, which knows when 47 + dma-mapping is (or is not) appropriate. 48 + - PP_FLAG_DMA_SYNC_DEV: netmem dma addr is not necessarily dma-syncable 49 + by the driver. The driver must delegate the dma syncing to the page_pool, 50 + which knows when dma-syncing is (or is not) appropriate. 51 + - PP_FLAG_ALLOW_UNREADABLE_NETMEM. The driver must specify this flag iff 52 + tcp-data-split is enabled. 53 + 54 + 5. The driver must not assume the netmem is readable and/or backed by pages. 55 + The netmem returned by the page_pool may be unreadable, in which case 56 + netmem_address() will return NULL. The driver must correctly handle 57 + unreadable netmem, i.e. don't attempt to handle its contents when 58 + netmem_address() is NULL. 59 + 60 + Ideally, drivers should not have to check the underlying netmem type via 61 + helpers like netmem_is_net_iov() or convert the netmem to any of its 62 + underlying types via netmem_to_page() or netmem_to_net_iov(). In most cases, 63 + netmem or page_pool helpers that abstract this complexity are provided 64 + (and more can be added). 65 + 66 + 6. The driver must use page_pool_dma_sync_netmem_for_cpu() in lieu of 67 + dma_sync_single_range_for_cpu(). For some memory providers, dma_syncing for 68 + CPU will be done by the page_pool, for others (particularly dmabuf memory 69 + provider), dma syncing for CPU is the responsibility of the userspace using 70 + dmabuf APIs. The driver must delegate the entire dma-syncing operation to 71 + the page_pool which will do it correctly. 72 + 73 + 7. Avoid implementing driver-specific recycling on top of the page_pool. Drivers 74 + cannot hold onto a struct page to do their own recycling as the netmem may 75 + not be backed by a struct page. However, you may hold onto a page_pool 76 + reference with page_pool_fragment_netmem() or page_pool_ref_netmem() for 77 + that purpose, but be mindful that some netmem types might have longer 78 + circulation times, such as when userspace holds a reference in zerocopy 79 + scenarios.