tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
media: admin-guide: add documentation file rkisp1.rst
Add the file 'rkisp1.rst' that documents the rkisp1 driver.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Acked-by: Helen Koike <helen.koike@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
authored by
Dafna Hirschfeld
and committed by
Mauro Carvalho Chehab
5cc0ebb2
409e975e
+200
+18
Documentation/admin-guide/media/rkisp1.dot
+18
Documentation/admin-guide/media/rkisp1.dot
···
1
+
digraph board {
2
+
rankdir=TB
3
+
n00000001 [label="{{<port0> 0 | <port1> 1} | rkisp1_isp\n/dev/v4l-subdev0 | {<port2> 2 | <port3> 3}}", shape=Mrecord, style=filled, fillcolor=green]
4
+
n00000001:port2 -> n00000006:port0
5
+
n00000001:port2 -> n00000009:port0
6
+
n00000001:port3 -> n00000014 [style=bold]
7
+
n00000006 [label="{{<port0> 0} | rkisp1_resizer_mainpath\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
8
+
n00000006:port1 -> n0000000c [style=bold]
9
+
n00000009 [label="{{<port0> 0} | rkisp1_resizer_selfpath\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
10
+
n00000009:port1 -> n00000010 [style=bold]
11
+
n0000000c [label="rkisp1_mainpath\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
12
+
n00000010 [label="rkisp1_selfpath\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
13
+
n00000014 [label="rkisp1_stats\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
14
+
n00000018 [label="rkisp1_params\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
15
+
n00000018 -> n00000001:port1 [style=bold]
16
+
n0000001c [label="{{} | imx219 4-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
17
+
n0000001c:port0 -> n00000001:port0
18
+
}
+181
Documentation/admin-guide/media/rkisp1.rst
+181
Documentation/admin-guide/media/rkisp1.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
.. include:: <isonum.txt>
4
+
5
+
=========================================
6
+
Rockchip Image Signal Processor (rkisp1)
7
+
=========================================
8
+
9
+
Introduction
10
+
============
11
+
12
+
This file documents the driver for the Rockchip ISP1 that is part of RK3288
13
+
and RK3399 SoCs. The driver is located under drivers/staging/media/rkisp1
14
+
and uses the Media-Controller API.
15
+
16
+
Topology
17
+
========
18
+
.. _rkisp1_topology_graph:
19
+
20
+
.. kernel-figure:: rkisp1.dot
21
+
:alt: Diagram of the default media pipeline topology
22
+
:align: center
23
+
24
+
25
+
The driver has 4 video devices:
26
+
27
+
- rkisp1_mainpath: capture device for retrieving images, usually in higher
28
+
resolution.
29
+
- rkisp1_selfpath: capture device for retrieving images.
30
+
- rkisp1_stats: a metadata capture device that sends statistics.
31
+
- rkisp1_params: a metadata output device that receives parameters
32
+
configurations from userspace.
33
+
34
+
The driver has 3 subdevices:
35
+
36
+
- rkisp1_resizer_mainpath: used to resize and downsample frames for the
37
+
mainpath capture device.
38
+
- rkisp1_resizer_selfpath: used to resize and downsample frames for the
39
+
selfpath capture device.
40
+
- rkisp1_isp: is connected to the sensor and is responsible for all the isp
41
+
operations.
42
+
43
+
44
+
rkisp1_mainpath, rkisp1_selfpath - Frames Capture Video Nodes
45
+
-------------------------------------------------------------
46
+
Those are the `mainpath` and `selfpath` capture devices to capture frames.
47
+
Those entities are the DMA engines that write the frames to memory.
48
+
The selfpath video device can capture YUV/RGB formats. Its input is YUV encoded
49
+
stream and it is able to convert it to RGB. The selfpath is not able to
50
+
capture bayer formats.
51
+
The mainpath can capture both bayer and YUV formats but it is not able to
52
+
capture RGB formats.
53
+
Both capture videos support
54
+
the ``V4L2_CAP_IO_MC`` :ref:`capability <device-capabilities>`.
55
+
56
+
57
+
rkisp1_resizer_mainpath, rkisp1_resizer_selfpath - Resizers Subdevices Nodes
58
+
----------------------------------------------------------------------------
59
+
Those are resizer entities for the mainpath and the selfpath. Those entities
60
+
can scale the frames up and down and also change the YUV sampling (for example
61
+
YUV4:2:2 -> YUV4:2:0). They also have cropping capability on the sink pad.
62
+
The resizers entities can only operate on YUV:4:2:2 format
63
+
(MEDIA_BUS_FMT_YUYV8_2X8).
64
+
The mainpath capture device supports capturing video in bayer formats. In that
65
+
case the resizer of the mainpath is set to 'bypass' mode - it just forward the
66
+
frame without operating on it.
67
+
68
+
rkisp1_isp - Image Signal Processing Subdevice Node
69
+
---------------------------------------------------
70
+
This is the isp entity. It is connected to the sensor on sink pad 0 and
71
+
receives the frames using the CSI-2 protocol. It is responsible of configuring
72
+
the CSI-2 protocol. It has a cropping capability on sink pad 0 that is
73
+
connected to the sensor and on source pad 2 connected to the resizer entities.
74
+
Cropping on sink pad 0 defines the image region from the sensor.
75
+
Cropping on source pad 2 defines the region for the Image Stabilizer (IS).
76
+
77
+
.. _rkisp1_stats:
78
+
79
+
rkisp1_stats - Statistics Video Node
80
+
------------------------------------
81
+
The statistics video node outputs the 3A (auto focus, auto exposure and auto
82
+
white balance) statistics, and also histogram statistics for the frames that
83
+
are being processed by the rkisp1 to userspace applications.
84
+
Using these data, applications can implement algorithms and re-parameterize
85
+
the driver through the rkisp_params node to improve image quality during a
86
+
video stream.
87
+
The buffer format is defined by struct :c:type:`rkisp1_stat_buffer`, and
88
+
userspace should set
89
+
:ref:`V4L2_META_FMT_RK_ISP1_STAT_3A <v4l2-meta-fmt-stat-rkisp1>` as the
90
+
dataformat.
91
+
92
+
.. _rkisp1_params:
93
+
94
+
rkisp1_params - Parameters Video Node
95
+
-------------------------------------
96
+
The rkisp1_params video node receives a set of parameters from userspace
97
+
to be applied to the hardware during a video stream, allowing userspace
98
+
to dynamically modify values such as black level, cross talk corrections
99
+
and others.
100
+
101
+
The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
102
+
userspace should set
103
+
:ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-params-rkisp1>` as the
104
+
dataformat.
105
+
106
+
107
+
Capturing Video Frames Example
108
+
==============================
109
+
110
+
In the following example, the sensor connected to pad 0 of 'rkisp1_isp' is
111
+
imx219.
112
+
113
+
The following commands can be used to capture video from the selfpath video
114
+
node with dimension 900x800 planar format YUV 4:2:2. It uses all cropping
115
+
capabilities possible, (see explanation right below)
116
+
117
+
.. code-block:: bash
118
+
119
+
# set the links
120
+
"media-ctl" "-d" "platform:rkisp1" "-r"
121
+
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx219 4-0010':0 -> 'rkisp1_isp':0 [1]"
122
+
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
123
+
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [0]"
124
+
125
+
# set format for imx219 4-0010:0
126
+
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx219 4-0010":0 [fmt:SRGGB10_1X10/1640x1232]'
127
+
128
+
# set format for rkisp1_isp pads:
129
+
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SRGGB10_1X10/1640x1232 crop: (0,0)/1600x1200]'
130
+
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1600x1200 crop: (0,0)/1500x1100]'
131
+
132
+
# set format for rkisp1_resizer_selfpath pads:
133
+
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1500x1100 crop: (300,400)/1400x1000]'
134
+
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1 [fmt:YUYV8_2X8/900x800]'
135
+
136
+
# set format for rkisp1_selfpath:
137
+
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=900,height=800,"
138
+
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "pixelformat=422P"
139
+
140
+
# start streaming:
141
+
v4l2-ctl "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "10"
142
+
143
+
144
+
In the above example the sensor is configured to bayer format:
145
+
`SRGGB10_1X10/1640x1232`. The rkisp1_isp:0 pad should be configured to the
146
+
same mbus format and dimensions as the sensor, otherwise streaming will fail
147
+
with 'EPIPE' error. So it is also configured to `SRGGB10_1X10/1640x1232`.
148
+
In addition, the rkisp1_isp:0 pad is configured to cropping `(0,0)/1600x1200`.
149
+
150
+
The cropping dimensions are automatically propagated to be the format of the
151
+
isp source pad `rkisp1_isp:2`. Another cropping operation is configured on
152
+
the isp source pad: `(0,0)/1500x1100`.
153
+
154
+
The resizer's sink pad `rkisp1_resizer_selfpath` should be configured to format
155
+
`YUYV8_2X8/1500x1100` in order to match the format on the other side of the
156
+
link. In addition a cropping `(300,400)/1400x1000` is configured on it.
157
+
158
+
The source pad of the resizer, `rkisp1_resizer_selfpath:1` is configured to
159
+
format `YUYV8_2X8/900x800`. That means that the resizer first crop a window
160
+
of `(300,400)/1400x100` from the received frame and then scales this window
161
+
to dimension `900x800`.
162
+
163
+
Note that the above example does not uses the stats-params control loop.
164
+
Therefore the capture frames will not go through the 3A algorithms and
165
+
probably won't have a good quality, and can even look dark and greenish.
166
+
167
+
Configuring Quantization
168
+
========================
169
+
170
+
The driver supports limited and full range quantization on YUV formats,
171
+
where limited is the default.
172
+
To switch between one or the other, userspace should use the Colorspace
173
+
Conversion API (CSC) for subdevices on source pad 2 of the
174
+
isp (`rkisp1_isp:2`). The quantization configured on this pad is the
175
+
quantization of the captured video frames on the mainpath and selfpath
176
+
video nodes.
177
+
Note that the resizer and capture entities will always report
178
+
``V4L2_QUANTIZATION_DEFAULT`` even if the quantization is configured to full
179
+
range on `rkisp1_isp:2`. So in order to get the configured quantization,
180
+
application should get it from pad `rkisp1_isp:2`.
181
+