Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

drm/rcar-du: Fix maximum frame buffer pitch computation

The maximum pitch constraint for the hardware is expressed in pixels.
Convert it to bytes to validate frame buffer creation, as frame buffer
pitches are expressed in bytes.

Reported-by: Phil Edworthy <phil.edworthy@renesas.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

authored by

Laurent Pinchart and committed by
Dave Airlie
8bed5cc7 e05444be

+11 -2
+11 -2
drivers/gpu/drm/rcar-du/rcar_du_kms.c
··· 135 135 { 136 136 struct rcar_du_device *rcdu = dev->dev_private; 137 137 const struct rcar_du_format_info *format; 138 + unsigned int max_pitch; 138 139 unsigned int align; 140 + unsigned int bpp; 139 141 140 142 format = rcar_du_format_info(mode_cmd->pixel_format); 141 143 if (format == NULL) { ··· 146 144 return ERR_PTR(-EINVAL); 147 145 } 148 146 147 + /* 148 + * The pitch and alignment constraints are expressed in pixels on the 149 + * hardware side and in bytes in the DRM API. 150 + */ 151 + bpp = format->planes == 2 ? 1 : format->bpp / 8; 152 + max_pitch = 4096 * bpp; 153 + 149 154 if (rcar_du_needs(rcdu, RCAR_DU_QUIRK_ALIGN_128B)) 150 155 align = 128; 151 156 else 152 - align = 16 * format->bpp / 8; 157 + align = 16 * bpp; 153 158 154 159 if (mode_cmd->pitches[0] & (align - 1) || 155 - mode_cmd->pitches[0] >= 8192) { 160 + mode_cmd->pitches[0] >= max_pitch) { 156 161 dev_dbg(dev->dev, "invalid pitch value %u\n", 157 162 mode_cmd->pitches[0]); 158 163 return ERR_PTR(-EINVAL);