(I'm so sorry!) Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

sched_ext: Place dsq_vtime next to dsq_priq

scx_dispatch_enqueue() calls rb_add() for a vtime-ordered DSQ while holding dsq->lock. At each level of the descent, scx_dsq_priq_less() loads the visited task's dsq_vtime, and that comparison selects rb_left or rb_right from the same task's dsq_priq. In the x86-64 benchmark configuration, these accesses fell on different 64-byte cache lines before this change, so a cold visited task could require a second cache line fill on the dependent descent path. Move dsq_vtime immediately before dsq_priq, keeping dsq_seq and dsq_flags next to dsq_list. In the two x86-64 layouts inspected, where scx starts at offsets 776 and 840 in task_struct, the fields from dsq_list through dsq_priq occupy one cache line. The exact cache line placement depends on the containing task_struct layout and is not guaranteed for every configuration or architecture. Where they share a line, avoiding the dependent cache line fill helps reduce dsq->lock hold time during vtime insertion and therefore contention on the lock. In a 16-vCPU KVM guest, the instrumented enqueue interval was 4-6% shorter with scx_lavd and scx_layered. scx_mitosis and end-to-end workload time showed no consistent change. This only reorders fields; no scheduling behavior change is intended. Signed-off-by: Usama Arif <usama.arif@linux.dev> Signed-off-by: Tejun Heo <tj@kernel.org>


Author Usama Arif Committer Tejun Heo Date Commit Parent e9b9ad3c
+12 -11
1 changed file
+12 -11
1 changed file