Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Documentation: media: mali-c55: Add Statistics documentation

Add documentation explaining the ability to capture statistics from
the mali-c55 driver's new V4L2 device, as well as the various tap
points from which those statistics can be drawn in the ISP's
processing flow. Additionally add a page detailing the new V4L2
meta format for the mali-c55 statistics.

Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>

authored by

Daniel Scally and committed by
Hans Verkuil
b35d9f3e 5d1f7403

+90 -1
+59 -1
Documentation/admin-guide/media/mali-c55.rst
··· 67 67 - `mali_c55 resizer fr`: The Full-Resolution pipe resizer 68 68 - `mali_c55 resizer ds`: The Downscale pipe resizer 69 69 70 - The driver has 2 V4L2 video devices: 70 + The driver has 3 V4L2 video devices: 71 71 72 72 - `mali-c55 fr`: The full-resolution pipe's capture device 73 73 - `mali-c55 ds`: The downscale pipe's capture device 74 + - `mali-c55 3a stats`: The 3A statistics capture device 74 75 75 76 Frame sequences are synchronised across to two capture devices, meaning if one 76 77 pipe is started later than the other the sequence numbers returned in its ··· 333 332 334 333 # Set format on the video device and stream 335 334 yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 335 + 336 + .. _mali-c55-3a-stats: 337 + 338 + Capturing ISP Statistics 339 + ======================== 340 + 341 + The ISP is capable of producing statistics for consumption by image processing 342 + algorithms running in userspace. These statistics can be captured by queueing 343 + buffers to the `mali-c55 3a stats` V4L2 Device whilst the ISP is streaming. Only 344 + the :ref:`V4L2_META_FMT_MALI_C55_STATS <v4l2-meta-fmt-mali-c55-stats>` 345 + format is supported, so no format-setting need be done: 346 + 347 + .. code-block:: none 348 + 349 + # We assume the media graph has been configured to support RGB565 capture 350 + # from the mali-c55 fr V4L2 Device, which is at /dev/video0. The statistics 351 + # V4L2 device is at /dev/video3 352 + 353 + yavta -f RGB565 -s 1920x1080 -c32 /dev/video0 && \ 354 + yavta -c10 -F /dev/video3 355 + 356 + The layout of the buffer is described by :c:type:`mali_c55_stats_buffer`, 357 + but broadly statistics are generated to support three image processing 358 + algorithms; AEXP (Auto-Exposure), AWB (Auto-White Balance) and AF (Auto-Focus). 359 + These stats can be drawn from various places in the Mali C55 ISP pipeline, known 360 + as "tap points". This high-level block diagram is intended to explain where in 361 + the processing flow the statistics can be drawn from:: 362 + 363 + +--> AEXP-2 +----> AEXP-1 +--> AF-0 364 + | +----> AF-1 | 365 + | | | 366 + +---------+ | +--------------+ | +--------------+ | 367 + | Input +-+-->+ Digital Gain +---+-->+ Black Level +---+---+ 368 + +---------+ +--------------+ +--------------+ | 369 + +-----------------------------------------------------------------+ 370 + | 371 + | +--------------+ +---------+ +----------------+ 372 + +-->| Sinter Noise +-+ White +--+--->| Lens Shading +--+---------------+ 373 + | Reduction | | Balance | | | | | | 374 + +--------------+ +---------+ | +----------------+ | | 375 + +---> AEXP-0 (A) +--> AEXP-0 (B) | 376 + +--------------------------------------------------------------------------+ 377 + | 378 + | +----------------+ +--------------+ +----------------+ 379 + +-->| Tone mapping +-+--->| Demosaicing +->+ Purple Fringe +-+-----------+ 380 + | | | +--------------+ | Correction | | | 381 + +----------------+ +-> AEXP-IRIDIX +----------------+ +---> AWB-0 | 382 + +----------------------------------------------------------------------------+ 383 + | +-------------+ +-------------+ 384 + +------------------->| Colour +---+--->| Output | 385 + | Correction | | | Pipelines | 386 + +-------------+ | +-------------+ 387 + +--> AWB-1 388 + 389 + At present all statistics are drawn from the 0th tap point for each algorithm; 390 + I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF 391 + statistics from AF-0. In the future this will be configurable. 336 392 337 393 References 338 394 ==========
+1
Documentation/userspace-api/media/v4l/meta-formats.rst
··· 13 13 .. toctree:: 14 14 :maxdepth: 1 15 15 16 + metafmt-arm-mali-c55 16 17 metafmt-c3-isp 17 18 metafmt-d4xx 18 19 metafmt-generic
+29
Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
··· 1 + .. SPDX-License-Identifier: GPL-2.0 2 + 3 + .. _v4l2-meta-fmt-mali-c55-stats: 4 + 5 + ************************************* 6 + V4L2_META_FMT_MALI_C55_STATS ('C55S') 7 + ************************************* 8 + 9 + 3A Statistics 10 + ============= 11 + 12 + The ISP device collects different statistics over an input bayer frame. Those 13 + statistics can be obtained by userspace from the 14 + :ref:`mali-c55 3a stats <mali-c55-3a-stats>` metadata capture video node, using 15 + the :c:type:`v4l2_meta_format` interface. The buffer contains a single instance 16 + of the C structure :c:type:`mali_c55_stats_buffer` defined in 17 + ``mali-c55-config.h``, so the structure can be obtained from the buffer by: 18 + 19 + .. code-block:: C 20 + 21 + struct mali_c55_stats_buffer *stats = 22 + (struct mali_c55_stats_buffer *)buf; 23 + 24 + For details of the statistics see :c:type:`mali_c55_stats_buffer`. 25 + 26 + Arm Mali-C55 uAPI data types 27 + ============================ 28 + 29 + .. kernel-doc:: include/uapi/linux/media/arm/mali-c55-config.h
+1
MAINTAINERS
··· 2115 2115 F: Documentation/admin-guide/media/mali-c55.rst 2116 2116 F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml 2117 2117 F: Documentation/userspace-api/media/drivers/mali-c55.rst 2118 + F: Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst 2118 2119 F: drivers/media/platform/arm/mali-c55/ 2119 2120 F: include/uapi/linux/media/arm/mali-c55-config.h 2120 2121