tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
media: Documentation: Split camera sensor documentation
Split camera sensor documentation into user and kernel portions. This
should make it easier for the user space developers to find the relevant
documentation.
Also add a list of exemplary drivers and add imx219 driver to it, besides
those that were already mentioned.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
authored by
Sakari Ailus
and committed by
Hans Verkuil
dc887661
2d21fef5
+147
-93
+38
-93
Documentation/driver-api/media/camera-sensor.rst
+38
-93
Documentation/driver-api/media/camera-sensor.rst
···
1
1
.. SPDX-License-Identifier: GPL-2.0
2
2
3
+
.. _media_writing_camera_sensor_drivers:
4
+
3
5
Writing camera sensor drivers
4
6
=============================
7
+
8
+
This document covers the in-kernel APIs only. For the best practices on
9
+
userspace API implementation in camera sensor drivers, please see
10
+
:ref:`media_using_camera_sensor_drivers`.
5
11
6
12
CSI-2 and parallel (BT.601 and BT.656) busses
7
13
---------------------------------------------
···
40
34
41
35
The preferred way to achieve this is using ``assigned-clocks``,
42
36
``assigned-clock-parents`` and ``assigned-clock-rates`` properties. See the
43
-
`clock device tree bindings <https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/clock/clock.yaml>`_
37
+
`clock device tree bindings
38
+
<https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/clock/clock.yaml>`_
44
39
for more information. The driver then gets the frequency using
45
40
``clk_get_rate()``.
46
41
···
92
85
a bridge driver, instead add runtime PM support to the sensor driver you are
93
86
using and drop its ``.s_power()`` handler.
94
87
95
-
See examples of runtime PM handling in e.g. ``drivers/media/i2c/ov8856.c`` and
96
-
``drivers/media/i2c/ccs/ccs-core.c``. The two drivers work in both ACPI and DT
97
-
based systems.
88
+
Please also see :ref:`examples <media-camera-sensor-examples>`.
98
89
99
90
Control framework
100
91
~~~~~~~~~~~~~~~~~
···
109
104
runtime PM was disabled, in either of which cases the driver may proceed to
110
105
access the device.
111
106
112
-
Frame size
113
-
----------
114
-
115
-
There are two distinct ways to configure the frame size produced by camera
116
-
sensors.
117
-
118
-
Freely configurable camera sensor drivers
119
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
120
-
121
-
Freely configurable camera sensor drivers expose the device's internal
122
-
processing pipeline as one or more sub-devices with different cropping and
123
-
scaling configurations. The output size of the device is the result of a series
124
-
of cropping and scaling operations from the device's pixel array's size.
125
-
126
-
An example of such a driver is the CCS driver (see ``drivers/media/i2c/ccs``).
127
-
128
-
Register list based drivers
129
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~
130
-
131
-
Register list based drivers generally, instead of able to configure the device
132
-
they control based on user requests, are limited to a number of preset
133
-
configurations that combine a number of different parameters that on hardware
134
-
level are independent. How a driver picks such configuration is based on the
135
-
format set on a source pad at the end of the device's internal pipeline.
136
-
137
-
Most sensor drivers are implemented this way, see e.g.
138
-
``drivers/media/i2c/imx319.c`` for an example.
139
-
140
-
Frame interval configuration
141
-
----------------------------
142
-
143
-
There are two different methods for obtaining possibilities for different frame
144
-
intervals as well as configuring the frame interval. Which one to implement
145
-
depends on the type of the device.
146
-
147
-
Raw camera sensors
148
-
~~~~~~~~~~~~~~~~~~
149
-
150
-
Instead of a high level parameter such as frame interval, the frame interval is
151
-
a result of the configuration of a number of camera sensor implementation
152
-
specific parameters. Luckily, these parameters tend to be the same for more or
153
-
less all modern raw camera sensors.
154
-
155
-
The frame interval is calculated using the following equation::
156
-
157
-
frame interval = (analogue crop width + horizontal blanking) *
158
-
(analogue crop height + vertical blanking) / pixel rate
159
-
160
-
The formula is bus independent and is applicable for raw timing parameters on
161
-
large variety of devices beyond camera sensors. Devices that have no analogue
162
-
crop, use the full source image size, i.e. pixel array size.
163
-
164
-
Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
165
-
``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
166
-
is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
167
-
the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
168
-
sub-device. The unit of that control is pixels per second.
169
-
170
-
Register list based drivers need to implement read-only sub-device nodes for the
171
-
purpose. Devices that are not register list based need these to configure the
172
-
device's internal processing pipeline.
173
-
174
-
The first entity in the linear pipeline is the pixel array. The pixel array may
175
-
be followed by other entities that are there to allow configuring binning,
176
-
skipping, scaling or digital crop :ref:`v4l2-subdev-selections`.
177
-
178
-
USB cameras etc. devices
179
-
~~~~~~~~~~~~~~~~~~~~~~~~
180
-
181
-
USB video class hardware, as well as many cameras offering a similar higher
182
-
level interface natively, generally use the concept of frame interval (or frame
183
-
rate) on device level in firmware or hardware. This means lower level controls
184
-
implemented by raw cameras may not be used on uAPI (or even kAPI) to control the
185
-
frame interval on these devices.
186
-
187
107
Rotation, orientation and flipping
188
108
----------------------------------
189
-
190
-
Some systems have the camera sensor mounted upside down compared to its natural
191
-
mounting rotation. In such cases, drivers shall expose the information to
192
-
userspace with the :ref:`V4L2_CID_CAMERA_SENSOR_ROTATION
193
-
<v4l2-camera-sensor-rotation>` control.
194
-
195
-
Sensor drivers shall also report the sensor's mounting orientation with the
196
-
:ref:`V4L2_CID_CAMERA_SENSOR_ORIENTATION <v4l2-camera-sensor-orientation>`.
197
109
198
110
Use ``v4l2_fwnode_device_parse()`` to obtain rotation and orientation
199
111
information from system firmware and ``v4l2_ctrl_new_fwnode_properties()`` to
200
112
register the appropriate controls.
201
113
202
-
Sensor drivers that have any vertical or horizontal flips embedded in the
203
-
register programming sequences shall initialize the V4L2_CID_HFLIP and
204
-
V4L2_CID_VFLIP controls with the values programmed by the register sequences.
205
-
The default values of these controls shall be 0 (disabled). Especially these
206
-
controls shall not be inverted, independently of the sensor's mounting
207
-
rotation.
114
+
.. _media-camera-sensor-examples:
115
+
116
+
Example drivers
117
+
---------------
118
+
119
+
Features implemented by sensor drivers vary, and depending on the set of
120
+
supported features and other qualities, particular sensor drivers better serve
121
+
the purpose of an example. The following drivers are known to be good examples:
122
+
123
+
.. flat-table:: Example sensor drivers
124
+
:header-rows: 0
125
+
:widths: 1 1 1 2
126
+
127
+
* - Driver name
128
+
- File(s)
129
+
- Driver type
130
+
- Example topic
131
+
* - CCS
132
+
- ``drivers/media/i2c/ccs/``
133
+
- Freely configurable
134
+
- Power management (ACPI and DT), UAPI
135
+
* - imx219
136
+
- ``drivers/media/i2c/imx219.c``
137
+
- Register list based
138
+
- Power management (DT), UAPI, mode selection
139
+
* - imx319
140
+
- ``drivers/media/i2c/imx319.c``
141
+
- Register list based
142
+
- Power management (ACPI and DT)
+104
Documentation/userspace-api/media/drivers/camera-sensor.rst
+104
Documentation/userspace-api/media/drivers/camera-sensor.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
.. _media_using_camera_sensor_drivers:
4
+
5
+
Using camera sensor drivers
6
+
===========================
7
+
8
+
This section describes common practices for how the V4L2 sub-device interface is
9
+
used to control the camera sensor drivers.
10
+
11
+
You may also find :ref:`media_writing_camera_sensor_drivers` useful.
12
+
13
+
Frame size
14
+
----------
15
+
16
+
There are two distinct ways to configure the frame size produced by camera
17
+
sensors.
18
+
19
+
Freely configurable camera sensor drivers
20
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
21
+
22
+
Freely configurable camera sensor drivers expose the device's internal
23
+
processing pipeline as one or more sub-devices with different cropping and
24
+
scaling configurations. The output size of the device is the result of a series
25
+
of cropping and scaling operations from the device's pixel array's size.
26
+
27
+
An example of such a driver is the CCS driver.
28
+
29
+
Register list based drivers
30
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
31
+
32
+
Register list based drivers generally, instead of able to configure the device
33
+
they control based on user requests, are limited to a number of preset
34
+
configurations that combine a number of different parameters that on hardware
35
+
level are independent. How a driver picks such configuration is based on the
36
+
format set on a source pad at the end of the device's internal pipeline.
37
+
38
+
Most sensor drivers are implemented this way.
39
+
40
+
Frame interval configuration
41
+
----------------------------
42
+
43
+
There are two different methods for obtaining possibilities for different frame
44
+
intervals as well as configuring the frame interval. Which one to implement
45
+
depends on the type of the device.
46
+
47
+
Raw camera sensors
48
+
~~~~~~~~~~~~~~~~~~
49
+
50
+
Instead of a high level parameter such as frame interval, the frame interval is
51
+
a result of the configuration of a number of camera sensor implementation
52
+
specific parameters. Luckily, these parameters tend to be the same for more or
53
+
less all modern raw camera sensors.
54
+
55
+
The frame interval is calculated using the following equation::
56
+
57
+
frame interval = (analogue crop width + horizontal blanking) *
58
+
(analogue crop height + vertical blanking) / pixel rate
59
+
60
+
The formula is bus independent and is applicable for raw timing parameters on
61
+
large variety of devices beyond camera sensors. Devices that have no analogue
62
+
crop, use the full source image size, i.e. pixel array size.
63
+
64
+
Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
65
+
``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
66
+
is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
67
+
the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
68
+
sub-device. The unit of that control is pixels per second.
69
+
70
+
Register list based drivers need to implement read-only sub-device nodes for the
71
+
purpose. Devices that are not register list based need these to configure the
72
+
device's internal processing pipeline.
73
+
74
+
The first entity in the linear pipeline is the pixel array. The pixel array may
75
+
be followed by other entities that are there to allow configuring binning,
76
+
skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
77
+
<VIDIOC_SUBDEV_G_SELECTION>`.
78
+
79
+
USB cameras etc. devices
80
+
~~~~~~~~~~~~~~~~~~~~~~~~
81
+
82
+
USB video class hardware, as well as many cameras offering a similar higher
83
+
level interface natively, generally use the concept of frame interval (or frame
84
+
rate) on device level in firmware or hardware. This means lower level controls
85
+
implemented by raw cameras may not be used on uAPI (or even kAPI) to control the
86
+
frame interval on these devices.
87
+
88
+
Rotation, orientation and flipping
89
+
----------------------------------
90
+
91
+
Some systems have the camera sensor mounted upside down compared to its natural
92
+
mounting rotation. In such cases, drivers shall expose the information to
93
+
userspace with the :ref:`V4L2_CID_CAMERA_SENSOR_ROTATION
94
+
<v4l2-camera-sensor-rotation>` control.
95
+
96
+
Sensor drivers shall also report the sensor's mounting orientation with the
97
+
:ref:`V4L2_CID_CAMERA_SENSOR_ORIENTATION <v4l2-camera-sensor-orientation>`.
98
+
99
+
Sensor drivers that have any vertical or horizontal flips embedded in the
100
+
register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP
101
+
<v4l2-cid-hflip>` and :ref:`V4L2_CID_VFLIP <v4l2-cid-vflip>` controls with the
102
+
values programmed by the register sequences. The default values of these
103
+
controls shall be 0 (disabled). Especially these controls shall not be inverted,
104
+
independently of the sensor's mounting rotation.
+1
Documentation/userspace-api/media/drivers/index.rst
+1
Documentation/userspace-api/media/drivers/index.rst
+4
Documentation/userspace-api/media/v4l/control.rst
+4
Documentation/userspace-api/media/v4l/control.rst
···
143
143
recognise the difference between digital and analogue gain use
144
144
controls ``V4L2_CID_DIGITAL_GAIN`` and ``V4L2_CID_ANALOGUE_GAIN``.
145
145
146
+
.. _v4l2-cid-hflip:
147
+
146
148
``V4L2_CID_HFLIP`` ``(boolean)``
147
149
Mirror the picture horizontally.
150
+
151
+
.. _v4l2-cid-vflip:
148
152
149
153
``V4L2_CID_VFLIP`` ``(boolean)``
150
154
Mirror the picture vertically.