tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Documentation: net: octeontx2: Add RVU HW and drivers overview
Added high level overview of OcteonTx2 RVU HW and functionality of
various drivers which will be upstreamed.
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
authored by
Sunil Goutham
and committed by
David S. Miller
493aeb26
6e92d71b
+161
+1
Documentation/networking/device_drivers/index.rst
+1
Documentation/networking/device_drivers/index.rst
+159
Documentation/networking/device_drivers/marvell/octeontx2.rst
+159
Documentation/networking/device_drivers/marvell/octeontx2.rst
···
1
+
.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
2
+
3
+
====================================
4
+
Marvell OcteonTx2 RVU Kernel Drivers
5
+
====================================
6
+
7
+
Copyright (c) 2020 Marvell International Ltd.
8
+
9
+
Contents
10
+
========
11
+
12
+
- `Overview`_
13
+
- `Drivers`_
14
+
- `Basic packet flow`_
15
+
16
+
Overview
17
+
========
18
+
19
+
Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC maps HW
20
+
resources from the network, crypto and other functional blocks into
21
+
PCI-compatible physical and virtual functions. Each functional block
22
+
again has multiple local functions (LFs) for provisioning to PCI devices.
23
+
RVU supports multiple PCIe SRIOV physical functions (PFs) and virtual
24
+
functions (VFs). PF0 is called the administrative / admin function (AF)
25
+
and has privileges to provision RVU functional block's LFs to each of the
26
+
PF/VF.
27
+
28
+
RVU managed networking functional blocks
29
+
- Network pool or buffer allocator (NPA)
30
+
- Network interface controller (NIX)
31
+
- Network parser CAM (NPC)
32
+
- Schedule/Synchronize/Order unit (SSO)
33
+
- Loopback interface (LBK)
34
+
35
+
RVU managed non-networking functional blocks
36
+
- Crypto accelerator (CPT)
37
+
- Scheduled timers unit (TIM)
38
+
- Schedule/Synchronize/Order unit (SSO)
39
+
Used for both networking and non networking usecases
40
+
41
+
Resource provisioning examples
42
+
- A PF/VF with NIX-LF & NPA-LF resources works as a pure network device
43
+
- A PF/VF with CPT-LF resource works as a pure crypto offload device.
44
+
45
+
RVU functional blocks are highly configurable as per software requirements.
46
+
47
+
Firmware setups following stuff before kernel boots
48
+
- Enables required number of RVU PFs based on number of physical links.
49
+
- Number of VFs per PF are either static or configurable at compile time.
50
+
Based on config, firmware assigns VFs to each of the PFs.
51
+
- Also assigns MSIX vectors to each of PF and VFs.
52
+
- These are not changed after kernel boot.
53
+
54
+
Drivers
55
+
=======
56
+
57
+
Linux kernel will have multiple drivers registering to different PF and VFs
58
+
of RVU. Wrt networking there will be 3 flavours of drivers.
59
+
60
+
Admin Function driver
61
+
---------------------
62
+
63
+
As mentioned above RVU PF0 is called the admin function (AF), this driver
64
+
supports resource provisioning and configuration of functional blocks.
65
+
Doesn't handle any I/O. It sets up few basic stuff but most of the
66
+
funcionality is achieved via configuration requests from PFs and VFs.
67
+
68
+
PF/VFs communicates with AF via a shared memory region (mailbox). Upon
69
+
receiving requests AF does resource provisioning and other HW configuration.
70
+
AF is always attached to host kernel, but PFs and their VFs may be used by host
71
+
kernel itself, or attached to VMs or to userspace applications like
72
+
DPDK etc. So AF has to handle provisioning/configuration requests sent
73
+
by any device from any domain.
74
+
75
+
AF driver also interacts with underlying firmware to
76
+
- Manage physical ethernet links ie CGX LMACs.
77
+
- Retrieve information like speed, duplex, autoneg etc
78
+
- Retrieve PHY EEPROM and stats.
79
+
- Configure FEC, PAM modes
80
+
- etc
81
+
82
+
From pure networking side AF driver supports following functionality.
83
+
- Map a physical link to a RVU PF to which a netdev is registered.
84
+
- Attach NIX and NPA block LFs to RVU PF/VF which provide buffer pools, RQs, SQs
85
+
for regular networking functionality.
86
+
- Flow control (pause frames) enable/disable/config.
87
+
- HW PTP timestamping related config.
88
+
- NPC parser profile config, basically how to parse pkt and what info to extract.
89
+
- NPC extract profile config, what to extract from the pkt to match data in MCAM entries.
90
+
- Manage NPC MCAM entries, upon request can frame and install requested packet forwarding rules.
91
+
- Defines receive side scaling (RSS) algorithms.
92
+
- Defines segmentation offload algorithms (eg TSO)
93
+
- VLAN stripping, capture and insertion config.
94
+
- SSO and TIM blocks config which provide packet scheduling support.
95
+
- Debugfs support, to check current resource provising, current status of
96
+
NPA pools, NIX RQ, SQ and CQs, various stats etc which helps in debugging issues.
97
+
- And many more.
98
+
99
+
Physical Function driver
100
+
------------------------
101
+
102
+
This RVU PF handles IO, is mapped to a physical ethernet link and this
103
+
driver registers a netdev. This supports SR-IOV. As said above this driver
104
+
communicates with AF with a mailbox. To retrieve information from physical
105
+
links this driver talks to AF and AF gets that info from firmware and responds
106
+
back ie cannot talk to firmware directly.
107
+
108
+
Supports ethtool for configuring links, RSS, queue count, queue size,
109
+
flow control, ntuple filters, dump PHY EEPROM, config FEC etc.
110
+
111
+
Virtual Function driver
112
+
-----------------------
113
+
114
+
There are two types VFs, VFs that share the physical link with their parent
115
+
SR-IOV PF and the VFs which work in pairs using internal HW loopback channels (LBK).
116
+
117
+
Type1:
118
+
- These VFs and their parent PF share a physical link and used for outside communication.
119
+
- VFs cannot communicate with AF directly, they send mbox message to PF and PF
120
+
forwards that to AF. AF after processing, responds back to PF and PF forwards
121
+
the reply to VF.
122
+
- From functionality point of view there is no difference between PF and VF as same type
123
+
HW resources are attached to both. But user would be able to configure few stuff only
124
+
from PF as PF is treated as owner/admin of the link.
125
+
126
+
Type2:
127
+
- RVU PF0 ie admin function creates these VFs and maps them to loopback block's channels.
128
+
- A set of two VFs (VF0 & VF1, VF2 & VF3 .. so on) works as a pair ie pkts sent out of
129
+
VF0 will be received by VF1 and viceversa.
130
+
- These VFs can be used by applications or virtual machines to communicate between them
131
+
without sending traffic outside. There is no switch present in HW, hence the support
132
+
for loopback VFs.
133
+
- These communicate directly with AF (PF0) via mbox.
134
+
135
+
Except for the IO channels or links used for packet reception and transmission there is
136
+
no other difference between these VF types. AF driver takes care of IO channel mapping,
137
+
hence same VF driver works for both types of devices.
138
+
139
+
Basic packet flow
140
+
=================
141
+
142
+
Ingress
143
+
-------
144
+
145
+
1. CGX LMAC receives packet.
146
+
2. Forwards the packet to the NIX block.
147
+
3. Then submitted to NPC block for parsing and then MCAM lookup to get the destination RVU device.
148
+
4. NIX LF attached to the destination RVU device allocates a buffer from RQ mapped buffer pool of NPA block LF.
149
+
5. RQ may be selected by RSS or by configuring MCAM rule with a RQ number.
150
+
6. Packet is DMA'ed and driver is notified.
151
+
152
+
Egress
153
+
------
154
+
155
+
1. Driver prepares a send descriptor and submits to SQ for transmission.
156
+
2. The SQ is already configured (by AF) to transmit on a specific link/channel.
157
+
3. The SQ descriptor ring is maintained in buffers allocated from SQ mapped pool of NPA block LF.
158
+
4. NIX block transmits the pkt on the designated channel.
159
+
5. NPC MCAM entries can be installed to divert pkt onto a different channel.
+1
MAINTAINERS
+1
MAINTAINERS