tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
media: admin-guide: Document the Raspberry Pi PiSP BE
Add documentation for the PiSP Back End memory-to-memory ISP.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
authored by
Jacopo Mondi
and committed by
Hans Verkuil
5b683b20
12187bd5
+130
+20
Documentation/admin-guide/media/raspberrypi-pisp-be.dot
+20
Documentation/admin-guide/media/raspberrypi-pisp-be.dot
···
1
+
digraph board {
2
+
rankdir=TB
3
+
n00000001 [label="{{<port0> 0 | <port1> 1 | <port2> 2 | <port7> 7} | pispbe\n | {<port3> 3 | <port4> 4 | <port5> 5 | <port6> 6}}", shape=Mrecord, style=filled, fillcolor=green]
4
+
n00000001:port3 -> n0000001c [style=bold]
5
+
n00000001:port4 -> n00000022 [style=bold]
6
+
n00000001:port5 -> n00000028 [style=bold]
7
+
n00000001:port6 -> n0000002e [style=bold]
8
+
n0000000a [label="pispbe-input\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
9
+
n0000000a -> n00000001:port0 [style=bold]
10
+
n00000010 [label="pispbe-tdn_input\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
11
+
n00000010 -> n00000001:port1 [style=bold]
12
+
n00000016 [label="pispbe-stitch_input\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
13
+
n00000016 -> n00000001:port2 [style=bold]
14
+
n0000001c [label="pispbe-output0\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
15
+
n00000022 [label="pispbe-output1\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
16
+
n00000028 [label="pispbe-tdn_output\n/dev/video5", shape=box, style=filled, fillcolor=yellow]
17
+
n0000002e [label="pispbe-stitch_output\n/dev/video6", shape=box, style=filled, fillcolor=yellow]
18
+
n00000034 [label="pispbe-config\n/dev/video7", shape=box, style=filled, fillcolor=yellow]
19
+
n00000034 -> n00000001:port7 [style=bold]
20
+
}
+109
Documentation/admin-guide/media/raspberrypi-pisp-be.rst
+109
Documentation/admin-guide/media/raspberrypi-pisp-be.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
=========================================================
4
+
Raspberry Pi PiSP Back End Memory-to-Memory ISP (pisp-be)
5
+
=========================================================
6
+
7
+
The PiSP Back End
8
+
=================
9
+
10
+
The PiSP Back End is a memory-to-memory Image Signal Processor (ISP) which reads
11
+
image data from DRAM memory and performs image processing as specified by the
12
+
application through the parameters in a configuration buffer, before writing
13
+
pixel data back to memory through two distinct output channels.
14
+
15
+
The ISP registers and programming model are documented in the `Raspberry Pi
16
+
Image Signal Processor (PiSP) Specification document`_
17
+
18
+
The PiSP Back End ISP processes images in tiles. The handling of image
19
+
tessellation and the computation of low-level configuration parameters is
20
+
realized by a free software library called `libpisp
21
+
<https://github.com/raspberrypi/libpisp>`_.
22
+
23
+
The full image processing pipeline, which involves capturing RAW Bayer data from
24
+
an image sensor through a MIPI CSI-2 compatible capture interface, storing them
25
+
in DRAM memory and processing them in the PiSP Back End to obtain images usable
26
+
by an application is implemented in `libcamera <https://libcamera.org>`_ as
27
+
part of the Raspberry Pi platform support.
28
+
29
+
The pisp-be driver
30
+
==================
31
+
32
+
The Raspberry Pi PiSP Back End (pisp-be) driver is located under
33
+
drivers/media/platform/raspberrypi/pisp-be. It uses the `V4L2 API` to register
34
+
a number of video capture and output devices, the `V4L2 subdev API` to register
35
+
a subdevice for the ISP that connects the video devices in a single media graph
36
+
realized using the `Media Controller (MC) API`.
37
+
38
+
The media topology registered by the `pisp-be` driver is represented below:
39
+
40
+
.. _pips-be-topology:
41
+
42
+
.. kernel-figure:: raspberrypi-pisp-be.dot
43
+
:alt: Diagram of the default media pipeline topology
44
+
:align: center
45
+
46
+
47
+
The media graph registers the following video device nodes:
48
+
49
+
- pispbe-input: output device for images to be submitted to the ISP for
50
+
processing.
51
+
- pispbe-tdn_input: output device for temporal denoise.
52
+
- pispbe-stitch_input: output device for image stitching (HDR).
53
+
- pispbe-output0: first capture device for processed images.
54
+
- pispbe-output1: second capture device for processed images.
55
+
- pispbe-tdn_output: capture device for temporal denoise.
56
+
- pispbe-stitch_output: capture device for image stitching (HDR).
57
+
- pispbe-config: output device for ISP configuration parameters.
58
+
59
+
pispbe-input
60
+
------------
61
+
62
+
Images to be processed by the ISP are queued to the `pispbe-input` output device
63
+
node. For a list of image formats supported as input to the ISP refer to the
64
+
`Raspberry Pi Image Signal Processor (PiSP) Specification document`_.
65
+
66
+
pispbe-tdn_input, pispbe-tdn_output
67
+
-----------------------------------
68
+
69
+
The `pispbe-tdn_input` output video device receives images to be processed by
70
+
the temporal denoise block which are captured from the `pispbe-tdn_output`
71
+
capture video device. Userspace is responsible for maintaining queues on both
72
+
devices, and ensuring that buffers completed on the output are queued to the
73
+
input.
74
+
75
+
pispbe-stitch_input, pispbe-stitch_output
76
+
-----------------------------------------
77
+
78
+
To realize HDR (high dynamic range) image processing the image stitching and
79
+
tonemapping blocks are used. The `pispbe-stitch_output` writes images to memory
80
+
and the `pispbe-stitch_input` receives the previously written frame to process
81
+
it along with the current input image. Userspace is responsible for maintaining
82
+
queues on both devices, and ensuring that buffers completed on the output are
83
+
queued to the input.
84
+
85
+
pispbe-output0, pispbe-output1
86
+
------------------------------
87
+
88
+
The two capture devices write to memory the pixel data as processed by the ISP.
89
+
90
+
pispbe-config
91
+
-------------
92
+
93
+
The `pispbe-config` output video devices receives a buffer of configuration
94
+
parameters that define the desired image processing to be performed by the ISP.
95
+
96
+
The format of the ISP configuration parameter is defined by
97
+
:c:type:`pisp_be_tiles_config` C structure and the meaning of each parameter is
98
+
described in the `Raspberry Pi Image Signal Processor (PiSP) Specification
99
+
document`_.
100
+
101
+
ISP configuration
102
+
=================
103
+
104
+
The ISP configuration is described solely by the content of the parameters
105
+
buffer. The only parameter that userspace needs to configure using the V4L2 API
106
+
is the image format on the output and capture video devices for validation of
107
+
the content of the parameters buffer.
108
+
109
+
.. _Raspberry Pi Image Signal Processor (PiSP) Specification document: https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf