tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Documentation: document adreno preemption
Add documentation about the preemption feature supported by the msm
driver.
Signed-off-by: Antonino Maniscalco <antomani103@gmail.com>
Patchwork: https://patchwork.freedesktop.org/patch/618032/
Signed-off-by: Rob Clark <robdclark@chromium.org>
authored by
Antonino Maniscalco
and committed by
Rob Clark
a20a91fb
20f33275
+99
+99
Documentation/gpu/msm-preemption.rst
+99
Documentation/gpu/msm-preemption.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
:orphan:
4
+
5
+
==============
6
+
MSM Preemption
7
+
==============
8
+
9
+
Preemption allows Adreno GPUs to switch to a higher priority ring when work is
10
+
pushed to it, reducing latency for high priority submissions.
11
+
12
+
When preemption is enabled 4 rings are initialized, corresponding to different
13
+
priority levels. Having multiple rings is purely a software concept as the GPU
14
+
only has registers to keep track of one graphics ring.
15
+
The kernel is able to switch which ring is currently being processed by
16
+
requesting preemption. When certain conditions are met, depending on the
17
+
priority level, the GPU will save its current state in a series of buffers,
18
+
then restores state from a similar set of buffers specified by the kernel. It
19
+
then resumes execution and fires an IRQ to let the kernel know the context
20
+
switch has completed.
21
+
22
+
This mechanism can be used by the kernel to switch between rings. Whenever a
23
+
submission occurs the kernel finds the highest priority ring which isn't empty
24
+
and preempts to it if said ring is not the one being currently executed. This is
25
+
also done whenever a submission completes to make sure execution resumes on a
26
+
lower priority ring when a higher priority ring is done.
27
+
28
+
Preemption levels
29
+
-----------------
30
+
31
+
Preemption can only occur at certain boundaries. The exact conditions can be
32
+
configured by changing the preemption level, this allows to compromise between
33
+
latency (ie. the time that passes between when the kernel requests preemption
34
+
and when the SQE begins saving state) and overhead (the amount of state that
35
+
needs to be saved).
36
+
37
+
The GPU offers 3 levels:
38
+
39
+
Level 0
40
+
Preemption only occurs at the submission level. This requires the least amount
41
+
of state to be saved as the execution of userspace submitted IBs is never
42
+
interrupted, however it offers very little benefit compared to not enabling
43
+
preemption of any kind.
44
+
45
+
Level 1
46
+
Preemption occurs at either bin level, if using GMEM rendering, or draw level
47
+
in the sysmem rendering case.
48
+
49
+
Level 2
50
+
Preemption occurs at draw level.
51
+
52
+
Level 1 is the mode that is used by the msm driver.
53
+
54
+
Additionally the GPU allows to specify a `skip_save_restore` option. This
55
+
disables the saving and restoring of all registers except those relating to the
56
+
operation of the SQE itself, reducing overhead. Saving and restoring is only
57
+
skipped when using GMEM with Level 1 preemption. When enabling this userspace is
58
+
expected to set the state that isn't preserved whenever preemption occurs which
59
+
is done by specifying preamble and postambles. Those are IBs that are executed
60
+
before and after preemption.
61
+
62
+
Preemption buffers
63
+
------------------
64
+
65
+
A series of buffers are necessary to store the state of rings while they are not
66
+
being executed. There are different kinds of preemption records and most of
67
+
those require one buffer per ring. This is because preemption never occurs
68
+
between submissions on the same ring, which always run in sequence when the ring
69
+
is active. This means that only one context per ring is effectively active.
70
+
71
+
SMMU_INFO
72
+
This buffer contains info about the current SMMU configuration such as the
73
+
ttbr0 register. The SQE firmware isn't actually able to save this record.
74
+
As a result SMMU info must be saved manually from the CP to a buffer and the
75
+
SMMU record updated with info from said buffer before triggering
76
+
preemption.
77
+
78
+
NON_SECURE
79
+
This is the main preemption record where most state is saved. It is mostly
80
+
opaque to the kernel except for the first few words that must be initialized
81
+
by the kernel.
82
+
83
+
SECURE
84
+
This saves state related to the GPU's secure mode.
85
+
86
+
NON_PRIV
87
+
The intended purpose of this record is unknown. The SQE firmware actually
88
+
ignores it and therefore msm doesn't handle it.
89
+
90
+
COUNTER
91
+
This record is used to save and restore performance counters.
92
+
93
+
Handling the permissions of those buffers is critical for security. All but the
94
+
NON_PRIV records need to be inaccessible from userspace, so they must be mapped
95
+
in the kernel address space with the MSM_BO_MAP_PRIV flag.
96
+
For example, making the NON_SECURE record accessible from userspace would allow
97
+
any process to manipulate a saved ring's RPTR which can be used to skip the
98
+
execution of some packets in a ring and execute user commands with higher
99
+
privileges.