Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

csky: Fixup raw_copy_from_user()

If raw_copy_from_user(to, from, N) returns K, callers expect
the first N - K bytes starting at to to have been replaced with
the contents of corresponding area starting at from and the last
K bytes of destination *left* *unmodified*.

What arch/sky/lib/usercopy.c is doing is broken - it can lead to e.g.
data corruption on write(2).

raw_copy_to_user() is inaccurate about return value, which is a bug,
but consequences are less drastic than for raw_copy_from_user().
And just what are those access_ok() doing in there? I mean, look into
linux/uaccess.h; that's where we do that check (as well as zero tail
on failure in the callers that need zeroing).

AFAICS, all of that shouldn't be hard to fix; something like a patch
below might make a useful starting point.

I would suggest moving these macros into usercopy.c (they are never
used anywhere else) and possibly expanding them there; if you leave
them alive, please at least rename __copy_user_zeroing(). Again,
it must not zero anything on failed read.

Said that, I'm not sure we won't be better off simply turning
usercopy.c into usercopy.S - all that is left there is a couple of
functions, each consisting only of inline asm.

Guo Ren reply:

Yes, raw_copy_from_user is wrong, it's no need zeroing code.

unsigned long _copy_from_user(void *to, const void __user *from,
unsigned long n)
{
unsigned long res = n;
might_fault();
if (likely(access_ok(from, n))) {
kasan_check_write(to, n);
res = raw_copy_from_user(to, from, n);
}
if (unlikely(res))
memset(to + (n - res), 0, res);
return res;
}
EXPORT_SYMBOL(_copy_from_user);

You are right and access_ok() should be removed.

but, how about:
do {
...
"2: stw %3, (%1, 0) \n" \
+ " subi %0, 4 \n" \
"9: stw %4, (%1, 4) \n" \
+ " subi %0, 4 \n" \
"10: stw %5, (%1, 8) \n" \
+ " subi %0, 4 \n" \
"11: stw %6, (%1, 12) \n" \
+ " subi %0, 4 \n" \
" addi %2, 16 \n" \
" addi %1, 16 \n" \

Don't expand __ex_table

AI Viro reply:

Hey, I've no idea about the instruction scheduling on csky -
if that doesn't slow the things down, all the better. It's just
that copy_to_user() and friends are on fairly hot codepaths,
and in quite a few situations they will dominate the speed of
e.g. read(2). So I tried to keep the fast path unchanged.
Up to the architecture maintainers, obviously. Which would be
you...

As for the fixups size increase (__ex_table size is unchanged)...
You have each of those macros expanded exactly once.
So the size is not a serious argument, IMO - useless complexity
would be, if it is, in fact, useless; the size... not really,
especially since those extra subi will at least offset it.

Again, up to you - asm optimizations of (essentially)
memcpy()-style loops are tricky and can depend upon the
fairly subtle details of architecture. So even on something
I know reasonably well I would resort to direct experiments
if I can't pass the buck to architecture maintainers.

It *is* worth optimizing - this is where read() from a file
that is already in page cache spends most of the time, etc.

Guo Ren reply:

Thx, after fixup some typo “sub %0, 4”, apply the patch.

TODO:
- user copy/from codes are still need optimizing.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>

authored by

Al Viro and committed by
Guo Ren
51bb38cb 67002814

+28 -29
+26 -23
arch/csky/include/asm/uaccess.h
··· 253 253 254 254 extern int __get_user_bad(void); 255 255 256 - #define __copy_user(to, from, n) \ 256 + #define ___copy_to_user(to, from, n) \ 257 257 do { \ 258 258 int w0, w1, w2, w3; \ 259 259 asm volatile( \ ··· 288 288 " subi %0, 4 \n" \ 289 289 " br 3b \n" \ 290 290 "5: cmpnei %0, 0 \n" /* 1B */ \ 291 - " bf 8f \n" \ 291 + " bf 13f \n" \ 292 292 " ldb %3, (%2, 0) \n" \ 293 293 "6: stb %3, (%1, 0) \n" \ 294 294 " addi %2, 1 \n" \ 295 295 " addi %1, 1 \n" \ 296 296 " subi %0, 1 \n" \ 297 297 " br 5b \n" \ 298 - "7: br 8f \n" \ 298 + "7: subi %0, 4 \n" \ 299 + "8: subi %0, 4 \n" \ 300 + "12: subi %0, 4 \n" \ 301 + " br 13f \n" \ 299 302 ".section __ex_table, \"a\" \n" \ 300 303 ".align 2 \n" \ 301 - ".long 2b, 7b \n" \ 302 - ".long 9b, 7b \n" \ 303 - ".long 10b, 7b \n" \ 304 + ".long 2b, 13f \n" \ 305 + ".long 4b, 13f \n" \ 306 + ".long 6b, 13f \n" \ 307 + ".long 9b, 12b \n" \ 308 + ".long 10b, 8b \n" \ 304 309 ".long 11b, 7b \n" \ 305 - ".long 4b, 7b \n" \ 306 - ".long 6b, 7b \n" \ 307 310 ".previous \n" \ 308 - "8: \n" \ 311 + "13: \n" \ 309 312 : "=r"(n), "=r"(to), "=r"(from), "=r"(w0), \ 310 313 "=r"(w1), "=r"(w2), "=r"(w3) \ 311 314 : "0"(n), "1"(to), "2"(from) \ 312 315 : "memory"); \ 313 316 } while (0) 314 317 315 - #define __copy_user_zeroing(to, from, n) \ 318 + #define ___copy_from_user(to, from, n) \ 316 319 do { \ 317 320 int tmp; \ 318 321 int nsave; \ ··· 358 355 " addi %1, 1 \n" \ 359 356 " subi %0, 1 \n" \ 360 357 " br 5b \n" \ 361 - "8: mov %3, %0 \n" \ 362 - " movi %4, 0 \n" \ 363 - "9: stb %4, (%1, 0) \n" \ 364 - " addi %1, 1 \n" \ 365 - " subi %3, 1 \n" \ 366 - " cmpnei %3, 0 \n" \ 367 - " bt 9b \n" \ 368 - " br 7f \n" \ 358 + "8: stw %3, (%1, 0) \n" \ 359 + " subi %0, 4 \n" \ 360 + " bf 7f \n" \ 361 + "9: subi %0, 8 \n" \ 362 + " bf 7f \n" \ 363 + "13: stw %3, (%1, 8) \n" \ 364 + " subi %0, 12 \n" \ 365 + " bf 7f \n" \ 369 366 ".section __ex_table, \"a\" \n" \ 370 367 ".align 2 \n" \ 371 - ".long 2b, 8b \n" \ 368 + ".long 2b, 7f \n" \ 369 + ".long 4b, 7f \n" \ 370 + ".long 6b, 7f \n" \ 372 371 ".long 10b, 8b \n" \ 373 - ".long 11b, 8b \n" \ 374 - ".long 12b, 8b \n" \ 375 - ".long 4b, 8b \n" \ 376 - ".long 6b, 8b \n" \ 372 + ".long 11b, 9b \n" \ 373 + ".long 12b,13b \n" \ 377 374 ".previous \n" \ 378 375 "7: \n" \ 379 376 : "=r"(n), "=r"(to), "=r"(from), "=r"(nsave), \
+2 -6
arch/csky/lib/usercopy.c
··· 7 7 unsigned long raw_copy_from_user(void *to, const void *from, 8 8 unsigned long n) 9 9 { 10 - if (access_ok(from, n)) 11 - __copy_user_zeroing(to, from, n); 12 - else 13 - memset(to, 0, n); 10 + ___copy_from_user(to, from, n); 14 11 return n; 15 12 } 16 13 EXPORT_SYMBOL(raw_copy_from_user); ··· 15 18 unsigned long raw_copy_to_user(void *to, const void *from, 16 19 unsigned long n) 17 20 { 18 - if (access_ok(to, n)) 19 - __copy_user(to, from, n); 21 + ___copy_to_user(to, from, n); 20 22 return n; 21 23 } 22 24 EXPORT_SYMBOL(raw_copy_to_user);