tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
1Automatically bind swap device to numa node
2-------------------------------------------
3
4If the system has more than one swap device and swap device has the node
5information, we can make use of this information to decide which swap
6device to use in get_swap_pages() to get better performance.
7
8
9How to use this feature
10-----------------------
11
12Swap device has priority and that decides the order of it to be used. To make
13use of automatically binding, there is no need to manipulate priority settings
14for swap devices. e.g. on a 2 node machine, assume 2 swap devices swapA and
15swapB, with swapA attached to node 0 and swapB attached to node 1, are going
16to be swapped on. Simply swapping them on by doing:
17# swapon /dev/swapA
18# swapon /dev/swapB
19
20Then node 0 will use the two swap devices in the order of swapA then swapB and
21node 1 will use the two swap devices in the order of swapB then swapA. Note
22that the order of them being swapped on doesn't matter.
23
24A more complex example on a 4 node machine. Assume 6 swap devices are going to
25be swapped on: swapA and swapB are attached to node 0, swapC is attached to
26node 1, swapD and swapE are attached to node 2 and swapF is attached to node3.
27The way to swap them on is the same as above:
28# swapon /dev/swapA
29# swapon /dev/swapB
30# swapon /dev/swapC
31# swapon /dev/swapD
32# swapon /dev/swapE
33# swapon /dev/swapF
34
35Then node 0 will use them in the order of:
36swapA/swapB -> swapC -> swapD -> swapE -> swapF
37swapA and swapB will be used in a round robin mode before any other swap device.
38
39node 1 will use them in the order of:
40swapC -> swapA -> swapB -> swapD -> swapE -> swapF
41
42node 2 will use them in the order of:
43swapD/swapE -> swapA -> swapB -> swapC -> swapF
44Similaly, swapD and swapE will be used in a round robin mode before any
45other swap devices.
46
47node 3 will use them in the order of:
48swapF -> swapA -> swapB -> swapC -> swapD -> swapE
49
50
51Implementation details
52----------------------
53
54The current code uses a priority based list, swap_avail_list, to decide
55which swap device to use and if multiple swap devices share the same
56priority, they are used round robin. This change here replaces the single
57global swap_avail_list with a per-numa-node list, i.e. for each numa node,
58it sees its own priority based list of available swap devices. Swap
59device's priority can be promoted on its matching node's swap_avail_list.
60
61The current swap device's priority is set as: user can set a >=0 value,
62or the system will pick one starting from -1 then downwards. The priority
63value in the swap_avail_list is the negated value of the swap device's
64due to plist being sorted from low to high. The new policy doesn't change
65the semantics for priority >=0 cases, the previous starting from -1 then
66downwards now becomes starting from -2 then downwards and -1 is reserved
67as the promoted value. So if multiple swap devices are attached to the same
68node, they will all be promoted to priority -1 on that node's plist and will
69be used round robin before any other swap devices.