Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

iio: dac: ad5592r: fix temperature channel scaling value

The scale value for the temperature channel is (assuming Vref=2.5 and
the datasheet):

376.7897513

When calculating both val and val2 for the temperature scale we
use (3767897513/25) and multiply it by Vref (here I assume 2500mV) to
obtain:

2500 * (3767897513/25) ==> 376789751300

Finally we divide with remainder by 10^9 to get:

val = 376
val2 = 789751300

However, we return IIO_VAL_INT_PLUS_MICRO (should have been NANO) as
the scale type. So when converting the raw temperature value to the
'processed' temperature value we will get (assuming raw=810,
offset=-753):

processed = (raw + offset) * scale_val
= (810 + -753) * 376
= 21432

processed += div((raw + offset) * scale_val2, 10^6)
+= div((810 + -753) * 789751300, 10^6)
+= 45015
==> 66447
==> 66.4 Celcius

instead of the expected 21.5 Celsius.

Fix this issue by changing IIO_VAL_INT_PLUS_MICRO to
IIO_VAL_INT_PLUS_NANO.

Fixes: 56ca9db862bf ("iio: dac: Add support for the AD5592R/AD5593R ADCs/DACs")
Signed-off-by: Marc Ferland <marc.ferland@sonatest.com>
Link: https://lore.kernel.org/r/20240501150554.1871390-1-marc.ferland@sonatest.com
Cc: <Stable@vger.kernel.org>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

authored by

Marc Ferland and committed by
Jonathan Cameron
279428df 0f0f6306

+1 -1
+1 -1
drivers/iio/dac/ad5592r-base.c
··· 415 415 s64 tmp = *val * (3767897513LL / 25LL); 416 416 *val = div_s64_rem(tmp, 1000000000LL, val2); 417 417 418 - return IIO_VAL_INT_PLUS_MICRO; 418 + return IIO_VAL_INT_PLUS_NANO; 419 419 } 420 420 421 421 mutex_lock(&st->lock);