Something went wrong. Try again.
(I'm so sorry!) Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Something went wrong. Try again.
11 kB · 357 lines
at master
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358.. SPDX-License-Identifier: GPL-2.0
=============CPU Isolation=============
Introduction============
"CPU Isolation" means leaving a CPU exclusive to a given workloadwithout any undesired code interference from the kernel.
Those interferences, commonly pointed out as "noise", can be triggeredby asynchronous events (interrupts, timers, scheduler preemption byworkqueues and kthreads, ...) or synchronous events (syscalls and pagefaults).
Such noise usually goes unnoticed. After all, synchronous events are acomponent of the requested kernel service. And asynchronous events areeither sufficiently well-distributed by the scheduler when executedas tasks or reasonably fast when executed as interrupt. The timerinterrupt can even execute 1024 times per seconds without a significantand measurable impact most of the time.
However some rare and extreme workloads can be quite sensitive tothose kinds of noise. This is the case, for example, with highbandwidth network processing that can't afford losing a single packetor very low latency network processing. Typically those use casesinvolve DPDK, bypassing the kernel networking stack and performingdirect access to the networking device from userspace.
In order to run a CPU without or with limited kernel noise, therelated housekeeping work needs to be either shut down, migrated oroffloaded.
Housekeeping============
In the CPU isolation terminology, housekeeping is the work, oftenasynchronous, that the kernel needs to process in order to maintainall its services. It matches the noises and disturbances enumeratedabove except when at least one CPU is isolated. Then housekeeping maymake use of further coping mechanisms if CPU-tied work must beoffloaded.
Housekeeping CPUs are the non-isolated CPUs where the kernel noiseis moved away from isolated CPUs.
The isolation can be implemented in several ways depending on thenature of the noise:
- Unbound work, where "unbound" means not tied to any CPU, can be simply migrated away from isolated CPUs to housekeeping CPUs. This is the case of unbound workqueues, kthreads and timers.
- Bound work, where "bound" means tied to a specific CPU, usually can't be moved away as-is by nature. Either:
- The work must switch to a locked implementation. E.g.: This is the case of RCU with CONFIG_RCU_NOCB_CPU.
- The related feature must be shut down and considered incompatible with isolated CPUs. E.g.: Lockup watchdog, unreliable clocksources, etc...
- An elaborate and heavyweight coping mechanism stands as a replacement. E.g.: the timer tick is shut down on nohz_full CPUs but with the constraint of running a single task on them. A significant cost penalty is added on kernel entry/exit and a residual 1Hz scheduler tick is offloaded to housekeeping CPUs.
In any case, housekeeping work has to be handled, which is why theremust be at least one housekeeping CPU in the system, preferably moreif the machine runs a lot of CPUs. For example one per node on NUMAsystems.
Also CPU isolation often means a tradeoff between noise-free isolatedCPUs and added overhead on housekeeping CPUs, sometimes even onisolated CPUs entering the kernel.
Isolation features==================
Different levels of isolation can be configured in the kernel, each ofwhich has its own drawbacks and tradeoffs.
Scheduler domain isolation--------------------------
This feature isolates a CPU from the scheduler topology. As a result,the target isn't part of the load balancing. Tasks won't migrateeither from or to it unless affined explicitly.
As a side effect the CPU is also isolated from unbound workqueues andunbound kthreads.
Requirements~~~~~~~~~~~~
- CONFIG_CPUSETS=y for the cpusets-based interface
Tradeoffs~~~~~~~~~
By nature, the system load is overall less distributed since some CPUsare extracted from the global load balancing.
Interfaces~~~~~~~~~~
- Documentation/admin-guide/cgroup-v2.rst cpuset isolated partitions are recommended because they are tunable at runtime.
- The 'isolcpus=' kernel boot parameter with the 'domain' flag is a less flexible alternative that doesn't allow for runtime reconfiguration.
IRQs isolation--------------
Isolate the IRQs whenever possible, so that they don't fire on thetarget CPUs.
Interfaces~~~~~~~~~~
- The file /proc/irq/\*/smp_affinity as explained in detail in Documentation/core-api/irq/irq-affinity.rst page.
- The "irqaffinity=" kernel boot parameter for a default setting.
- The "managed_irq" flag in the "isolcpus=" kernel boot parameter tries a best effort affinity override for managed IRQs.
Full Dynticks (aka nohz_full)-----------------------------
Full dynticks extends the dynticks idle mode, which stops the tick whenthe CPU is idle, to CPUs running a single task in userspace. That is,the timer tick is stopped if the environment allows it.
Global timer callbacks are also isolated from the nohz_full CPUs.
Requirements~~~~~~~~~~~~
- CONFIG_NO_HZ_FULL=y
Constraints~~~~~~~~~~~
- The isolated CPUs must run a single task only. Multitask requires the tick to maintain preemption. This is usually fine since the workload usually can't stand the latency of random context switches.
- No call to the kernel from isolated CPUs, at the risk of triggering random noise.
- No use of POSIX CPU timers on isolated CPUs.
- Architecture must have a stable and reliable clocksource (no unreliable TSC that requires the watchdog).
Tradeoffs~~~~~~~~~
In terms of cost, this is the most invasive isolation feature. It isassumed to be used when the workload spends most of its time inuserspace and doesn't rely on the kernel except for preparatorywork because:
- RCU adds more overhead due to the locked, offloaded and threaded callbacks processing (the same that would be obtained with "rcu_nocbs" boot parameter).
- Kernel entry/exit through syscalls, exceptions and IRQs are more costly due to fully ordered RmW operations that maintain userspace as RCU extended quiescent state. Also the CPU time is accounted on kernel boundaries instead of periodically from the tick.
- Housekeeping CPUs must run a 1Hz residual remote scheduler tick on behalf of the isolated CPUs.
Checklist=========
You have set up each of the above isolation features but you stillobserve jitters that trash your workload? Make sure to check a fewelements before proceeding.
Some of these checklist items are similar to those of real-timeworkloads:
- Use mlock() to prevent your pages from being swapped away. Page faults are usually not compatible with jitter sensitive workloads.
- Avoid SMT to prevent your hardware thread from being "preempted" by another one.
- CPU frequency changes may induce subtle sorts of jitter in a workload. Cpufreq should be used and tuned with caution.
- Deep C-states may result in latency issues upon wake-up. If this happens to be a problem, C-states can be limited via kernel boot parameters such as processor.max_cstate or intel_idle.max_cstate. More finegrained tunings are described in Documentation/admin-guide/pm/cpuidle.rst page
- Your system may be subject to firmware-originating interrupts - x86 has System Management Interrupts (SMIs) for example. Check your system BIOS to disable such interference, and with some luck your vendor will have a BIOS tuning guidance for low-latency operations.
Full isolation example======================
In this example, the system has 8 CPUs and the 8th is to be fullyisolated. Since CPUs start from 0, the 8th CPU is CPU 7.
Kernel parameters-----------------
Set the following kernel boot parameters to disable SMT and setup tickand IRQ isolation:
- Full dynticks: nohz_full=7
- IRQs isolation: irqaffinity=0-6
- Managed IRQs isolation: isolcpus=managed_irq,7
- Prevent SMT: nosmt
The full command line is then:
nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt
CPUSET configuration (cgroup v2)--------------------------------
Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following scriptisolates CPU 7 from scheduler domains.
::
cd /sys/fs/cgroup # Activate the cpuset subsystem echo +cpuset > cgroup.subtree_control # Create partition to be isolated mkdir test cd test echo +cpuset > cgroup.subtree_control # Isolate CPU 7 echo 7 > cpuset.cpus echo "isolated" > cpuset.cpus.partition
The userspace workload----------------------
Fake a pure userspace workload, the program below runs a dummyuserspace loop on the isolated CPU 7.
::
#include <stdio.h> #include <fcntl.h> #include <unistd.h> #include <errno.h> int main(void) { // Move the current task to the isolated cpuset (bind to CPU 7) int fd = open("/sys/fs/cgroup/test/cgroup.procs", O_WRONLY); if (fd < 0) { perror("Can't open cpuset file...\n"); return 0; }
write(fd, "0\n", 2); close(fd);
// Run an endless dummy loop until the launcher kills us while (1) ;
return 0; }
Build it and save for later step:
::
# gcc user_loop.c -o user_loop
The launcher------------
The below launcher runs the above program for 10 seconds and tracesthe noise resulting from preempting tasks and IRQs.
::
TRACING=/sys/kernel/tracing/ # Make sure tracing is off for now echo 0 > $TRACING/tracing_on # Flush previous traces echo > $TRACING/trace # Record disturbance from other tasks echo 1 > $TRACING/events/sched/sched_switch/enable # Record disturbance from interrupts echo 1 > $TRACING/events/irq_vectors/enable # Now we can start tracing echo 1 > $TRACING/tracing_on # Run the dummy user_loop for 10 seconds on CPU 7 ./user_loop & USER_LOOP_PID=$! sleep 10 kill $USER_LOOP_PID # Disable tracing and save traces from CPU 7 in a file echo 0 > $TRACING/tracing_on cat $TRACING/per_cpu/cpu7/trace > trace.7
If no specific problem arose, the output of trace.7 should look likethe following:
::
<idle>-0 [007] d..2. 1980.976624: sched_switch: prev_comm=swapper/7 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=user_loop next_pid=1553 next_prio=120 user_loop-1553 [007] d.h.. 1990.946593: reschedule_entry: vector=253 user_loop-1553 [007] d.h.. 1990.946593: reschedule_exit: vector=253
That is, no specific noise triggered between the first trace and thesecond during 10 seconds when user_loop was running.
Debugging=========
Of course things are never so easy, especially on this matter.Chances are that actual noise will be observed in the aforementionedtrace.7 file.
The best way to investigate further is to enable finer grainedtracepoints such as those of subsystems producing asynchronousevents: workqueue, timer, irq_vector, etc... It also can beinteresting to enable the tick_stop event to diagnose why the tick isretained when that happens.
Some tools may also be useful for higher level analysis:
- Documentation/tools/rtla/rtla.rst provides a suite of tools to analyze latency and noise in the system. For example Documentation/tools/rtla/rtla-osnoise.rst runs a kernel tracer that analyzes and output a summary of the noises.
- cpunoise does something similar to rtla-osnoise but in userspace. It is available at https://git.kernel.org/pub/scm/linux/kernel/git/frederic/cpunoise.git