Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

drivers/tty/vt: use umin() instead of min_t(u16, ...) for row/col limits

The row/column bounds (for a screen window box) are changed from
'offset one' to 'offset zero' and bound to the screen size using:
v->xs = min_t(u16, v->xs - 1, vc->vc_cols - 1);
This has the side effect of converting zero to the limit.

A check I'm adding to min_t() reports that (u16)(v->xs - 1) (etc)
discards signiticant bits (because v->xs is promoted to 'int' before
the addition).
If v->xs is zero (it comes from userspace) it converts -1 to 0xffff.
This is then bounded to 'vc->vc_cols - 1' which will be fine.

Replace with:
v->xs = umin(v->xs - 1, vc->vc_cols - 1);
which again converts a -1 to unsigned - this time to 0xffffffff,
with the same overall effect.

Whether zero is meant to mean the 'maximum size' is unknown.
I can't find any documentation for the ioctl and it pre-dates git.

Detected by an extra check added to min_t().

Signed-off-by: David Laight <david.laight.linux@gmail.com>
Reviewed-by: Jiri Slaby <jirislaby@kernel.org>
Link: https://patch.msgid.link/20251119224140.8616-28-david.laight.linux@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

authored by

David Laight and committed by
Greg Kroah-Hartman
6c84a61a 719f3df3

+5 -4
+5 -4
drivers/tty/vt/selection.c
··· 348 348 return 0; 349 349 } 350 350 351 - v->xs = min_t(u16, v->xs - 1, vc->vc_cols - 1); 352 - v->ys = min_t(u16, v->ys - 1, vc->vc_rows - 1); 353 - v->xe = min_t(u16, v->xe - 1, vc->vc_cols - 1); 354 - v->ye = min_t(u16, v->ye - 1, vc->vc_rows - 1); 351 + /* Historically 0 => max value */ 352 + v->xs = umin(v->xs - 1, vc->vc_cols - 1); 353 + v->ys = umin(v->ys - 1, vc->vc_rows - 1); 354 + v->xe = umin(v->xe - 1, vc->vc_cols - 1); 355 + v->ye = umin(v->ye - 1, vc->vc_rows - 1); 355 356 356 357 if (mouse_reporting() && (v->sel_mode & TIOCL_SELMOUSEREPORT)) { 357 358 mouse_report(tty, v->sel_mode & TIOCL_SELBUTTONMASK, v->xs,