tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
media: Documentation: add documentation of Intel IPU6 driver and hardware overview
Add a documentation for an overview of IPU6 hardware and describe the main
the components of IPU6 driver.
Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
authored by
Bingbu Cao
and committed by
Hans Verkuil
d06fc8b6
ba124c8c
+206
+1
Documentation/driver-api/media/drivers/index.rst
+1
Documentation/driver-api/media/drivers/index.rst
+205
Documentation/driver-api/media/drivers/ipu6.rst
+205
Documentation/driver-api/media/drivers/ipu6.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
==================
4
+
Intel IPU6 Driver
5
+
==================
6
+
7
+
Author: Bingbu Cao <bingbu.cao@intel.com>
8
+
9
+
Overview
10
+
=========
11
+
12
+
Intel IPU6 is the sixth generation of Intel Image Processing Unit used in some
13
+
Intel Chipsets such as Tiger Lake, Jasper Lake, Alder Lake, Raptor Lake and
14
+
Meteor Lake. IPU6 consists of two major systems: Input System (ISYS) and
15
+
Processing System (PSYS). IPU6 are visible on the PCI bus as a single device, it
16
+
can be found by ``lspci``:
17
+
18
+
``0000:00:05.0 Multimedia controller: Intel Corporation Device xxxx (rev xx)``
19
+
20
+
IPU6 has a 16 MB BAR in PCI configuration Space for MMIO registers which is
21
+
visible for driver.
22
+
23
+
Buttress
24
+
=========
25
+
26
+
The IPU6 is connecting to the system fabric with Buttress which is enabling host
27
+
driver to control the IPU6, it also allows IPU6 access the system memory to
28
+
store and load frame pixel streams and any other metadata.
29
+
30
+
Buttress mainly manages several system functionalities: power management,
31
+
interrupt handling, firmware authentication and global timer sync.
32
+
33
+
ISYS and PSYS Power flow
34
+
------------------------
35
+
36
+
IPU6 driver initialize the ISYS and PSYS power up or down request by setting the
37
+
Buttress frequency control register for ISYS and PSYS
38
+
(``IPU6_BUTTRESS_REG_IS_FREQ_CTL`` and ``IPU6_BUTTRESS_REG_PS_FREQ_CTL``) in
39
+
function:
40
+
41
+
.. c:function:: int ipu6_buttress_power(...)
42
+
43
+
Buttress forwards the request to Punit, after Punit execute the power up flow,
44
+
Buttress indicates driver that ISYS or PSYS is powered up by updating the power
45
+
status registers.
46
+
47
+
.. Note:: ISYS power up needs take place prior to PSYS power up, ISYS power down
48
+
needs take place after PSYS power down due to hardware limitation.
49
+
50
+
Interrupt
51
+
---------
52
+
53
+
IPU6 interrupt can be generated as MSI or INTA, interrupt will be triggered when
54
+
ISYS, PSYS, Buttress event or error happen, driver can get the interrupt cause
55
+
by reading the interrupt status register ``BUTTRESS_REG_ISR_STATUS``, driver
56
+
clears the irq status and then calls specific ISYS or PSYS irq handler.
57
+
58
+
.. c:function:: irqreturn_t ipu6_buttress_isr(int irq, ...)
59
+
60
+
Security and firmware authentication
61
+
-------------------------------------
62
+
63
+
To address the IPU6 firmware security concerns, the IPU6 firmware needs to
64
+
undergo an authentication process before it is allowed to executed on the IPU6
65
+
internal processors. The IPU6 driver will work with Converged Security Engine
66
+
(CSE) to complete authentication process. The CSE is responsible of
67
+
authenticating the IPU6 firmware. The authenticated firmware binary is copied
68
+
into an isolated memory region. Firmware authentication process is implemented
69
+
by CSE following an IPC handshake with the IPU6 driver. There are some Buttress
70
+
registers used by the CSE and the IPU6 driver to communicate with each other via
71
+
IPC.
72
+
73
+
.. c:function:: int ipu6_buttress_authenticate(...)
74
+
75
+
Global timer sync
76
+
-----------------
77
+
78
+
The IPU6 driver initiates a Hammock Harbor synchronization flow each time it
79
+
starts camera operation. The IPU6 will synchronizes an internal counter in the
80
+
Buttress with a copy of the SoC time, this counter maintains the up-to-date time
81
+
until camera operation is stopped. The IPU6 driver can use this time counter to
82
+
calibrate the timestamp based on the timestamp in response event from firmware.
83
+
84
+
.. c:function:: int ipu6_buttress_start_tsc_sync(...)
85
+
86
+
DMA and MMU
87
+
============
88
+
89
+
The IPU6 has its own scalar processor where the firmware run at and an internal
90
+
32-bit virtual address space. The IPU6 has MMU address translation hardware to
91
+
allow that scalar processors to access the internal memory and external system
92
+
memory through IPU6 virtual address. The address translation is based on two
93
+
levels of page lookup tables stored in system memory which are maintained by the
94
+
IPU6 driver. The IPU6 driver sets the level-1 page table base address to MMU
95
+
register and allows MMU to perform page table lookups.
96
+
97
+
The IPU6 driver exports its own DMA operations. The IPU6 driver will update the
98
+
page table entries for each DMA operation and invalidate the MMU TLB after each
99
+
unmap and free.
100
+
101
+
.. code-block:: none
102
+
103
+
const struct dma_map_ops ipu6_dma_ops = {
104
+
.alloc = ipu6_dma_alloc,
105
+
.free = ipu6_dma_free,
106
+
.mmap = ipu6_dma_mmap,
107
+
.map_sg = ipu6_dma_map_sg,
108
+
.unmap_sg = ipu6_dma_unmap_sg,
109
+
...
110
+
};
111
+
112
+
.. Note:: IPU6 MMU works behind IOMMU so for each IPU6 DMA ops, driver will call
113
+
generic PCI DMA ops to ask IOMMU to do the additional mapping if VT-d
114
+
enabled.
115
+
116
+
Firmware file format
117
+
====================
118
+
119
+
The IPU6 firmware is in Code Partition Directory (CPD) file format. The CPD
120
+
firmware contains a CPD header, several CPD entries and components. The CPD
121
+
component includes 3 entries - manifest, metadata and module data. Manifest and
122
+
metadata are defined by CSE and used by CSE for authentication. Module data is
123
+
specific to IPU6 which holds the binary data of firmware called package
124
+
directory. The IPU6 driver (``ipu6-cpd.c`` in particular) parses and validates
125
+
the CPD firmware file and gets the package directory binary data of the IPU6
126
+
firmware, copies it to specific DMA buffer and sets its base address to Buttress
127
+
``FW_SOURCE_BASE`` register. Finally the CSE will do authentication for this
128
+
firmware binary.
129
+
130
+
131
+
Syscom interface
132
+
================
133
+
134
+
The IPU6 driver communicates with firmware via the Syscom ABI. Syscom is an
135
+
inter-processor communication mechanism between the IPU scalar processors and
136
+
the CPU. There are a number of resources shared between firmware and software.
137
+
A system memory region where the message queues reside, firmware can access the
138
+
memory region via the IPU MMU. The Syscom queues are FIFO fixed depth queues
139
+
with a configurable number of tokens (messages). There are also common IPU6 MMIO
140
+
registers where the queue read and write indices reside. Software and firmware
141
+
function as producer and consumer of tokens in the queues and update the write
142
+
and read indices separately when sending or receiving each message.
143
+
144
+
The IPU6 driver must prepare and configure the number of input and output
145
+
queues, configure the count of tokens per queue and the size of per token before
146
+
initiating and starting the communication with firmware. Firmware and software
147
+
must use same configurations. The IPU6 Buttress has a number of firmware boot
148
+
parameter registers which can be used to store the address of configuration and
149
+
initialise the Syscom state, then driver can request firmware to start and run via
150
+
setting the scalar processor control status register.
151
+
152
+
Input System
153
+
============
154
+
155
+
IPU6 input system consists of MIPI D-PHY and several CSI-2 receivers. It can
156
+
capture image pixel data from camera sensors or other MIPI CSI-2 output devices.
157
+
158
+
D-PHYs and CSI-2 ports lane mapping
159
+
-----------------------------------
160
+
161
+
The IPU6 integrates different D-PHY IPs on different SoCs, on Tiger Lake and
162
+
Alder Lake, IPU6 integrates MCD10 D-PHY, IPU6SE on Jasper Lake integrates JSL
163
+
D-PHY and IPU6EP on Meteor Lake integrates a Synopsys DWC D-PHY. There is an
164
+
adaptional layer between D-PHY and CSI-2 receiver controller which includes port
165
+
configuration, PHY wrapper or private test interfaces for D-PHY. There are 3
166
+
D-PHY drivers ``ipu6-isys-mcd-phy.c``, ``ipu6-isys-jsl-phy.c`` and
167
+
``ipu6-isys-dwc-phy.c`` program the above 3 D-PHYs in IPU6.
168
+
169
+
Different IPU6 versions have different D-PHY lanes mappings, On Tiger Lake,
170
+
there are 12 data lanes and 8 clock lanes, IPU6 support maximum 8 CSI-2 ports,
171
+
see the PPI mmapping in ``ipu6-isys-mcd-phy.c`` for more information. On Jasper
172
+
Lake and Alder Lake, D-PHY has 8 data lanes and 4 clock lanes, the IPU6 supports
173
+
maximum 4 CSI-2 ports. For Meteor Lake, D-PHY has 12 data lanes and 6 clock
174
+
lanes so IPU6 support maximum 6 CSI-2 ports.
175
+
176
+
.. Note:: Each pair of CSI-2 two ports is a single unit that can share the data
177
+
lanes. For example, for CSI-2 port 0 and 1, CSI-2 port 0 support
178
+
maximum 4 data lanes, CSI-2 port 1 support maximum 2 data lanes, CSI-2
179
+
port 0 with 2 data lanes can work together with CSI-2 port 1 with 2
180
+
data lanes. If trying to use CSI-2 port 0 with 4 lanes, CSI-2 port 1
181
+
will not be available as the 4 data lanes are shared by CSI-2 port 0
182
+
and 1. The same applies to CSI ports 2/3, 4/5 and 7/8.
183
+
184
+
ISYS firmware ABIs
185
+
------------------
186
+
187
+
The IPU6 firmware implements a series of ABIs for software access. In general,
188
+
software firstly prepares the stream configuration ``struct
189
+
ipu6_fw_isys_stream_cfg_data_abi`` and sends the configuration to firmware via
190
+
sending ``STREAM_OPEN`` command. Stream configuration includes input pins and
191
+
output pins, input pin ``struct ipu6_fw_isys_input_pin_info_abi`` defines the
192
+
resolution and data type of input source, output pin ``struct
193
+
ipu6_fw_isys_output_pin_info_abi`` defines the output resolution, stride and
194
+
frame format, etc.
195
+
196
+
Once the driver gets the interrupt from firmware that indicates stream open
197
+
successfully, the driver will send the ``STREAM_START`` and ``STREAM_CAPTURE``
198
+
command to request firmware to start capturing image frames. ``STREAM_CAPTURE``
199
+
command queues the buffers to firmware with ``struct
200
+
ipu6_fw_isys_frame_buff_set``, software then waits for the interrupt and
201
+
response from firmware, ``PIN_DATA_READY`` means a buffer is ready on a specific
202
+
output pin and then software can return the buffer to user.
203
+
204
+
.. Note:: See :ref:`Examples<ipu6_isys_capture_examples>` about how to do
205
+
capture by IPU6 ISYS driver.