tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Documentation: gpu: nova-core: Document fwsec operation and layout
Add explanation of fwsec with diagrams. This helps clarify how the
nova-core falcon boot works.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://lore.kernel.org/r/20250708-nova-docs-v4-7-9d188772c4c7@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
authored by
Joel Fernandes
and committed by
Danilo Krummrich
e5e716db
952e6224
+182
+181
Documentation/gpu/nova/core/fwsec.rst
+181
Documentation/gpu/nova/core/fwsec.rst
···
1
+
.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)
2
+
3
+
=========================
4
+
FWSEC (Firmware Security)
5
+
=========================
6
+
This document briefly/conceptually describes the FWSEC (Firmware Security) image
7
+
and its role in the GPU boot sequence. As such, this information is subject to
8
+
change in the future and is only current as of the Ampere GPU family. However,
9
+
hopefully the concepts described will be useful for understanding the kernel code
10
+
that deals with it. All the information is derived from publicly available
11
+
sources such as public drivers and documentation.
12
+
13
+
The role of FWSEC is to provide a secure boot process. It runs in
14
+
'Heavy-secure' mode, and performs firmware verification after a GPU reset
15
+
before loading various ucode images onto other microcontrollers on the GPU,
16
+
such as the PMU and GSP.
17
+
18
+
FWSEC itself is an application stored in the VBIOS ROM in the FWSEC partition of
19
+
ROM (see vbios.rst for more details). It contains different commands like FRTS
20
+
(Firmware Runtime Services) and SB (Secure Booting other microcontrollers after
21
+
reset and loading them with other non-FWSEC ucode). The kernel driver only needs
22
+
to perform FRTS, since Secure Boot (SB) has already completed by the time the driver
23
+
is loaded.
24
+
25
+
The FRTS command carves out the WPR2 region (Write protected region) which contains
26
+
data required for power management. Once setup, only HS mode ucode can access it
27
+
(see falcon.rst for privilege levels).
28
+
29
+
The FWSEC image is located in the VBIOS ROM in the partition of the ROM that contains
30
+
various ucode images (also known as applications) -- one of them being FWSEC. For how
31
+
it is extracted, see vbios.rst and the vbios.rs source code.
32
+
33
+
The Falcon data for each ucode images (including the FWSEC image) is a combination
34
+
of headers, data sections (DMEM) and instruction code sections (IMEM). All these
35
+
ucode images are stored in the same ROM partition and the PMU table is used to look
36
+
up the application to load it based on its application ID (see vbios.rs).
37
+
38
+
For the nova-core driver, the FWSEC contains an 'application interface' called
39
+
DMEMMAPPER. This interface is used to execute the 'FWSEC-FRTS' command, among others.
40
+
For Ampere, FWSEC is running on the GSP in Heavy-secure mode and runs FRTS.
41
+
42
+
FWSEC Memory Layout
43
+
-------------------
44
+
The memory layout of the FWSEC image is as follows::
45
+
46
+
+---------------------------------------------------------------+
47
+
| FWSEC ROM image (type 0xE0) |
48
+
| |
49
+
| +---------------------------------+ |
50
+
| | PMU Falcon Ucode Table | |
51
+
| | (PmuLookupTable) | |
52
+
| | +-------------------------+ | |
53
+
| | | Table Header | | |
54
+
| | | - version: 0x01 | | |
55
+
| | | - header_size: 6 | | |
56
+
| | | - entry_size: 6 | | |
57
+
| | | - entry_count: N | | |
58
+
| | | - desc_version:3(unused)| | |
59
+
| | +-------------------------+ | |
60
+
| | ... | |
61
+
| | +-------------------------+ | |
62
+
| | | Entry for FWSEC (0x85) | | |
63
+
| | | (PmuLookupTableEntry) | | |
64
+
| | | - app_id: 0x85 (FWSEC) |----|----+ |
65
+
| | | - target_id: 0x01 (PMU) | | | |
66
+
| | | - data: offset ---------|----|----|---+ look up FWSEC |
67
+
| | +-------------------------+ | | | |
68
+
| +---------------------------------+ | | |
69
+
| | | |
70
+
| | | |
71
+
| +---------------------------------+ | | |
72
+
| | FWSEC Ucode Component |<---+ | |
73
+
| | (aka Falcon data) | | |
74
+
| | +-------------------------+ | | |
75
+
| | | FalconUCodeDescV3 |<---|--------+ |
76
+
| | | - hdr | | |
77
+
| | | - stored_size | | |
78
+
| | | - pkc_data_offset | | |
79
+
| | | - interface_offset -----|----|----------------+ |
80
+
| | | - imem_phys_base | | | |
81
+
| | | - imem_load_size | | | |
82
+
| | | - imem_virt_base | | | |
83
+
| | | - dmem_phys_base | | | |
84
+
| | | - dmem_load_size | | | |
85
+
| | | - engine_id_mask | | | |
86
+
| | | - ucode_id | | | |
87
+
| | | - signature_count | | look up sig | |
88
+
| | | - signature_versions --------------+ | |
89
+
| | +-------------------------+ | | | |
90
+
| | (no gap) | | | |
91
+
| | +-------------------------+ | | | |
92
+
| | | Signatures Section |<---|-----+ | |
93
+
| | | (384 bytes per sig) | | | |
94
+
| | | - RSA-3K Signature 1 | | | |
95
+
| | | - RSA-3K Signature 2 | | | |
96
+
| | | ... | | | |
97
+
| | +-------------------------+ | | |
98
+
| | | | |
99
+
| | +-------------------------+ | | |
100
+
| | | IMEM Section (Code) | | | |
101
+
| | | | | | |
102
+
| | | Contains instruction | | | |
103
+
| | | code etc. | | | |
104
+
| | +-------------------------+ | | |
105
+
| | | | |
106
+
| | +-------------------------+ | | |
107
+
| | | DMEM Section (Data) | | | |
108
+
| | | | | | |
109
+
| | | +---------------------+ | | | |
110
+
| | | | Application | |<---|----------------+ |
111
+
| | | | Interface Table | | | |
112
+
| | | | (FalconAppifHdrV1) | | | |
113
+
| | | | Header: | | | |
114
+
| | | | - version: 0x01 | | | |
115
+
| | | | - header_size: 4 | | | |
116
+
| | | | - entry_size: 8 | | | |
117
+
| | | | - entry_count: N | | | |
118
+
| | | | | | | |
119
+
| | | | Entries: | | | |
120
+
| | | | +-----------------+ | | | |
121
+
| | | | | DEVINIT (ID 1) | | | | |
122
+
| | | | | - id: 0x01 | | | | |
123
+
| | | | | - dmemOffset X -|-|-|----+ |
124
+
| | | | +-----------------+ | | | |
125
+
| | | | +-----------------+ | | | |
126
+
| | | | | DMEMMAPPER(ID 4)| | | | |
127
+
| | | | | - id: 0x04 | | | | Used only for DevInit |
128
+
| | | | | (NVFW_FALCON_ | | | | application (not FWSEC) |
129
+
| | | | | APPIF_ID_DMEMMAPPER) | |
130
+
| | | | | - dmemOffset Y -|-|-|----|-----+ |
131
+
| | | | +-----------------+ | | | | |
132
+
| | | +---------------------+ | | | |
133
+
| | | | | | |
134
+
| | | +---------------------+ | | | |
135
+
| | | | DEVINIT Engine |<|----+ | Used by FWSEC |
136
+
| | | | Interface | | | | app. |
137
+
| | | +---------------------+ | | | |
138
+
| | | | | | |
139
+
| | | +---------------------+ | | | |
140
+
| | | | DMEM Mapper (ID 4) |<|----+-----+ |
141
+
| | | | (FalconAppifDmemmapperV3) | |
142
+
| | | | - signature: "DMAP" | | | |
143
+
| | | | - version: 0x0003 | | | |
144
+
| | | | - Size: 64 bytes | | | |
145
+
| | | | - cmd_in_buffer_off | |----|------------+ |
146
+
| | | | - cmd_in_buffer_size| | | | |
147
+
| | | | - cmd_out_buffer_off| |----|------------|-----+ |
148
+
| | | | - cmd_out_buffer_sz | | | | | |
149
+
| | | | - init_cmd | | | | | |
150
+
| | | | - features | | | | | |
151
+
| | | | - cmd_mask0/1 | | | | | |
152
+
| | | +---------------------+ | | | | |
153
+
| | | | | | | |
154
+
| | | +---------------------+ | | | | |
155
+
| | | | Command Input Buffer|<|----|------------+ | |
156
+
| | | | - Command data | | | | |
157
+
| | | | - Arguments | | | | |
158
+
| | | +---------------------+ | | | |
159
+
| | | | | | |
160
+
| | | +---------------------+ | | | |
161
+
| | | | Command Output |<|----|------------------+ |
162
+
| | | | Buffer | | | |
163
+
| | | | - Results | | | |
164
+
| | | | - Status | | | |
165
+
| | | +---------------------+ | | |
166
+
| | +-------------------------+ | |
167
+
| +---------------------------------+ |
168
+
| |
169
+
+---------------------------------------------------------------+
170
+
171
+
.. note::
172
+
This is using an GA-102 Ampere GPU as an example and could vary for future GPUs.
173
+
174
+
.. note::
175
+
The FWSEC image also plays a role in memory scrubbing (ECC initialization) and VPR
176
+
(Video Protected Region) initialization as well. Before the nova-core driver is even
177
+
loaded, the FWSEC image is running on the GSP in heavy-secure mode. After the devinit
178
+
sequence completes, it does VRAM memory scrubbing (ECC initialization). On consumer
179
+
GPUs, it scrubs only part of memory and then initiates 'async scrubbing'. Before this
180
+
async scrubbing completes, the unscrubbed VRAM cannot be used for allocation (thus DRM
181
+
memory allocators need to wait for this scrubbing to complete).