ARC: atomics: unbork atomic_fetch_##op()

In 4.19-rc1, Eugeniy reported weird boot and IO errors on ARC HSDK

| INFO: task syslogd:77 blocked for more than 10 seconds.
| Not tainted 4.19.0-rc1-00007-gf213acea4e88 #40
| "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
| message.
| syslogd D 0 77 76 0x00000000
|
| Stack Trace:
| __switch_to+0x0/0xac
| __schedule+0x1b2/0x730
| io_schedule+0x5c/0xc0
| __lock_page+0x98/0xdc
| find_lock_entry+0x38/0x100
| shmem_getpage_gfp.isra.3+0x82/0xbfc
| shmem_fault+0x46/0x138
| handle_mm_fault+0x5bc/0x924
| do_page_fault+0x100/0x2b8
| ret_from_exception+0x0/0x8

He bisected to 84c6591103db ("locking/atomics,
asm-generic/bitops/lock.h: Rewrite using atomic_fetch_*()")

This commit however only unmasked the real issue introduced by commit
4aef66c8ae9 ("locking/atomic, arch/arc: Fix build") which missed the
retry-if-scond-failed branch in atomic_fetch_##op() macros.

The bisected commit started using atomic_fetch_##op() macros for building
the rest of atomics.

Fixes: 4aef66c8ae9 ("locking/atomic, arch/arc: Fix build")
Reported-by: Eugeniy Paltsev <[e-mail]>
Acked-by: Peter Zijlstra (Intel) <[e-mail]>
Signed-off-by: Will Deacon <[e-mail]>
Signed-off-by: Vineet Gupta <[e-mail]>
[vgupta: wrote changelog]

arch/arc/include/asm/atomic.h

···
 	"1:	llock   %[orig], [%[ctr]]		\n"		\
 	"	" #asm_op " %[val], %[orig], %[i]	\n"		\
 	"	scond   %[val], [%[ctr]]		\n"		\
-	"						\n"		\
+	"	bnz     1b				\n"		\
 	: [val]	"=&r"	(val),						\
 	  [orig] "=&r" (orig)						\
 	: [ctr]	"r"	(&v->counter),					\