Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

math-emu: fix floating-point to integer unsigned saturation

The math-emu macros _FP_TO_INT and _FP_TO_INT_ROUND are supposed to
saturate their results for out-of-range arguments, except in the case
rsigned == 2 (when instead the low bits of the result are taken).
However, in the case rsigned == 0 (converting to unsigned integers),
they mistakenly produce 0 for positive results and the maximum
unsigned integer for negative results, the opposite of correct
unsigned saturation. This patch fixes the logic.

Signed-off-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Scott Wood <scottwood@freescale.com>

authored by

Joseph Myers and committed by
Scott Wood
4f6db5ef 28414a6d

+2 -2
+2 -2
include/math-emu/op-common.h
··· 685 685 else \ 686 686 { \ 687 687 r = 0; \ 688 - if (X##_s) \ 688 + if (!X##_s) \ 689 689 r = ~r; \ 690 690 } \ 691 691 FP_SET_EXCEPTION(FP_EX_INVALID); \ ··· 762 762 if (!rsigned) \ 763 763 { \ 764 764 r = 0; \ 765 - if (X##_s) \ 765 + if (!X##_s) \ 766 766 r = ~r; \ 767 767 } \ 768 768 else if (rsigned != 2) \