Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

membarrier: Create Documentation/scheduler/membarrier.rst

To gather the architecture requirements of the "private/global
expedited" membarrier commands. The file will be expanded to
integrate further information about the membarrier syscall (as
needed/desired in the future). While at it, amend some related
inline comments in the membarrier codebase.

Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/r/20240131144936.29190-3-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

authored by

Andrea Parri and committed by
Palmer Dabbelt
a14d11a0 d6cfd177

+51 -5
+1
Documentation/scheduler/index.rst
··· 7 7 8 8 9 9 completion 10 + membarrier 10 11 sched-arch 11 12 sched-bwc 12 13 sched-deadline
+39
Documentation/scheduler/membarrier.rst
··· 1 + .. SPDX-License-Identifier: GPL-2.0 2 + 3 + ======================== 4 + membarrier() System Call 5 + ======================== 6 + 7 + MEMBARRIER_CMD_{PRIVATE,GLOBAL}_EXPEDITED - Architecture requirements 8 + ===================================================================== 9 + 10 + Memory barriers before updating rq->curr 11 + ---------------------------------------- 12 + 13 + The commands MEMBARRIER_CMD_PRIVATE_EXPEDITED and MEMBARRIER_CMD_GLOBAL_EXPEDITED 14 + require each architecture to have a full memory barrier after coming from 15 + user-space, before updating rq->curr. This barrier is implied by the sequence 16 + rq_lock(); smp_mb__after_spinlock() in __schedule(). The barrier matches a full 17 + barrier in the proximity of the membarrier system call exit, cf. 18 + membarrier_{private,global}_expedited(). 19 + 20 + Memory barriers after updating rq->curr 21 + --------------------------------------- 22 + 23 + The commands MEMBARRIER_CMD_PRIVATE_EXPEDITED and MEMBARRIER_CMD_GLOBAL_EXPEDITED 24 + require each architecture to have a full memory barrier after updating rq->curr, 25 + before returning to user-space. The schemes providing this barrier on the various 26 + architectures are as follows. 27 + 28 + - alpha, arc, arm, hexagon, mips rely on the full barrier implied by 29 + spin_unlock() in finish_lock_switch(). 30 + 31 + - arm64 relies on the full barrier implied by switch_to(). 32 + 33 + - powerpc, riscv, s390, sparc, x86 rely on the full barrier implied by 34 + switch_mm(), if mm is not NULL; they rely on the full barrier implied 35 + by mmdrop(), otherwise. On powerpc and riscv, switch_mm() relies on 36 + membarrier_arch_switch_mm(). 37 + 38 + The barrier matches a full barrier in the proximity of the membarrier system call 39 + entry, cf. membarrier_{private,global}_expedited().
+1
MAINTAINERS
··· 14039 14039 M: "Paul E. McKenney" <paulmck@kernel.org> 14040 14040 L: linux-kernel@vger.kernel.org 14041 14041 S: Supported 14042 + F: Documentation/scheduler/membarrier.rst 14042 14043 F: arch/*/include/asm/membarrier.h 14043 14044 F: include/uapi/linux/membarrier.h 14044 14045 F: kernel/sched/membarrier.c
+6 -1
kernel/sched/core.c
··· 6638 6638 * if (signal_pending_state()) if (p->state & @state) 6639 6639 * 6640 6640 * Also, the membarrier system call requires a full memory barrier 6641 - * after coming from user-space, before storing to rq->curr. 6641 + * after coming from user-space, before storing to rq->curr; this 6642 + * barrier matches a full barrier in the proximity of the membarrier 6643 + * system call exit. 6642 6644 */ 6643 6645 rq_lock(rq, &rf); 6644 6646 smp_mb__after_spinlock(); ··· 6718 6716 * architectures where spin_unlock is a full barrier, 6719 6717 * - switch_to() for arm64 (weakly-ordered, spin_unlock 6720 6718 * is a RELEASE barrier), 6719 + * 6720 + * The barrier matches a full barrier in the proximity of 6721 + * the membarrier system call entry. 6721 6722 */ 6722 6723 ++*switch_count; 6723 6724
+4 -4
kernel/sched/membarrier.c
··· 251 251 return 0; 252 252 253 253 /* 254 - * Matches memory barriers around rq->curr modification in 254 + * Matches memory barriers after rq->curr modification in 255 255 * scheduler. 256 256 */ 257 257 smp_mb(); /* system call entry is not a mb. */ ··· 300 300 301 301 /* 302 302 * Memory barrier on the caller thread _after_ we finished 303 - * waiting for the last IPI. Matches memory barriers around 303 + * waiting for the last IPI. Matches memory barriers before 304 304 * rq->curr modification in scheduler. 305 305 */ 306 306 smp_mb(); /* exit from system call is not a mb */ ··· 339 339 return 0; 340 340 341 341 /* 342 - * Matches memory barriers around rq->curr modification in 342 + * Matches memory barriers after rq->curr modification in 343 343 * scheduler. 344 344 */ 345 345 smp_mb(); /* system call entry is not a mb. */ ··· 415 415 416 416 /* 417 417 * Memory barrier on the caller thread _after_ we finished 418 - * waiting for the last IPI. Matches memory barriers around 418 + * waiting for the last IPI. Matches memory barriers before 419 419 * rq->curr modification in scheduler. 420 420 */ 421 421 smp_mb(); /* exit from system call is not a mb */