tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
PCI/TPH: Add TPH documentation
Add a document for the TPH feature, including description of "notph" kernel
parameter and the API interface.
Co-developed-by: Eric Van Tassell <Eric.VanTassell@amd.com>
Link: https://lore.kernel.org/r/20241002165954.128085-4-wei.huang2@amd.com
Signed-off-by: Eric Van Tassell <Eric.VanTassell@amd.com>
Signed-off-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
authored by
Wei Huang
and committed by
Bjorn Helgaas
48d0fd2b
d2e8a348
+136
+1
Documentation/PCI/index.rst
+1
Documentation/PCI/index.rst
+132
Documentation/PCI/tph.rst
+132
Documentation/PCI/tph.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
4
+
===========
5
+
TPH Support
6
+
===========
7
+
8
+
:Copyright: 2024 Advanced Micro Devices, Inc.
9
+
:Authors: - Eric van Tassell <eric.vantassell@amd.com>
10
+
- Wei Huang <wei.huang2@amd.com>
11
+
12
+
13
+
Overview
14
+
========
15
+
16
+
TPH (TLP Processing Hints) is a PCIe feature that allows endpoint devices
17
+
to provide optimization hints for requests that target memory space.
18
+
These hints, in a format called Steering Tags (STs), are embedded in the
19
+
requester's TLP headers, enabling the system hardware, such as the Root
20
+
Complex, to better manage platform resources for these requests.
21
+
22
+
For example, on platforms with TPH-based direct data cache injection
23
+
support, an endpoint device can include appropriate STs in its DMA
24
+
traffic to specify which cache the data should be written to. This allows
25
+
the CPU core to have a higher probability of getting data from cache,
26
+
potentially improving performance and reducing latency in data
27
+
processing.
28
+
29
+
30
+
How to Use TPH
31
+
==============
32
+
33
+
TPH is presented as an optional extended capability in PCIe. The Linux
34
+
kernel handles TPH discovery during boot, but it is up to the device
35
+
driver to request TPH enablement if it is to be utilized. Once enabled,
36
+
the driver uses the provided API to obtain the Steering Tag for the
37
+
target memory and to program the ST into the device's ST table.
38
+
39
+
Enable TPH support in Linux
40
+
---------------------------
41
+
42
+
To support TPH, the kernel must be built with the CONFIG_PCIE_TPH option
43
+
enabled.
44
+
45
+
Manage TPH
46
+
----------
47
+
48
+
To enable TPH for a device, use the following function::
49
+
50
+
int pcie_enable_tph(struct pci_dev *pdev, int mode);
51
+
52
+
This function enables TPH support for device with a specific ST mode.
53
+
Current supported modes include:
54
+
55
+
* PCI_TPH_ST_NS_MODE - NO ST Mode
56
+
* PCI_TPH_ST_IV_MODE - Interrupt Vector Mode
57
+
* PCI_TPH_ST_DS_MODE - Device Specific Mode
58
+
59
+
`pcie_enable_tph()` checks whether the requested mode is actually
60
+
supported by the device before enabling. The device driver can figure out
61
+
which TPH mode is supported and can be properly enabled based on the
62
+
return value of `pcie_enable_tph()`.
63
+
64
+
To disable TPH, use the following function::
65
+
66
+
void pcie_disable_tph(struct pci_dev *pdev);
67
+
68
+
Manage ST
69
+
---------
70
+
71
+
Steering Tags are platform specific. PCIe spec does not specify where STs
72
+
are from. Instead PCI Firmware Specification defines an ACPI _DSM method
73
+
(see the `Revised _DSM for Cache Locality TPH Features ECN
74
+
<https://members.pcisig.com/wg/PCI-SIG/document/15470>`_) for retrieving
75
+
STs for a target memory of various properties. This method is what is
76
+
supported in this implementation.
77
+
78
+
To retrieve a Steering Tag for a target memory associated with a specific
79
+
CPU, use the following function::
80
+
81
+
int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type type,
82
+
unsigned int cpu_uid, u16 *tag);
83
+
84
+
The `type` argument is used to specify the memory type, either volatile
85
+
or persistent, of the target memory. The `cpu_uid` argument specifies the
86
+
CPU where the memory is associated to.
87
+
88
+
After the ST value is retrieved, the device driver can use the following
89
+
function to write the ST into the device::
90
+
91
+
int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index,
92
+
u16 tag);
93
+
94
+
The `index` argument is the ST table entry index the ST tag will be
95
+
written into. `pcie_tph_set_st_entry()` will figure out the proper
96
+
location of ST table, either in the MSI-X table or in the TPH Extended
97
+
Capability space, and write the Steering Tag into the ST entry pointed by
98
+
the `index` argument.
99
+
100
+
It is completely up to the driver to decide how to use these TPH
101
+
functions. For example a network device driver can use the TPH APIs above
102
+
to update the Steering Tag when interrupt affinity of a RX/TX queue has
103
+
been changed. Here is a sample code for IRQ affinity notifier:
104
+
105
+
.. code-block:: c
106
+
107
+
static void irq_affinity_notified(struct irq_affinity_notify *notify,
108
+
const cpumask_t *mask)
109
+
{
110
+
struct drv_irq *irq;
111
+
unsigned int cpu_id;
112
+
u16 tag;
113
+
114
+
irq = container_of(notify, struct drv_irq, affinity_notify);
115
+
cpumask_copy(irq->cpu_mask, mask);
116
+
117
+
/* Pick a right CPU as the target - here is just an example */
118
+
cpu_id = cpumask_first(irq->cpu_mask);
119
+
120
+
if (pcie_tph_get_cpu_st(irq->pdev, TPH_MEM_TYPE_VM, cpu_id,
121
+
&tag))
122
+
return;
123
+
124
+
if (pcie_tph_set_st_entry(irq->pdev, irq->msix_nr, tag))
125
+
return;
126
+
}
127
+
128
+
Disable TPH system-wide
129
+
-----------------------
130
+
131
+
There is a kernel command line option available to control TPH feature:
132
+
* "notph": TPH will be disabled for all endpoint devices.