tjh.dev
Clone of https://github.com/NixOS/nixpkgs.git (to stress-test knotserver)
1# What is this for?
2
3NixOS's traditional initrd is generated by listing the paths that
4should be included in initrd and copying the full runtime closure of
5those paths into the archive. For most things, like almost any
6executable, this involves copying the entirety of huge packages like
7glibc, when only things like the shared library files are needed. To
8solve this, NixOS does a variety of patchwork to edit the files being
9copied in so they only refer to small, patched up paths. For instance,
10executables and their shared library dependencies are copied into an
11`extraUtils` derivation, and every ELF file is patched to refer to
12files in that output.
13
14The problem with this is that it is often difficult to correctly patch
15some things. For instance, systemd bakes the path to the `mount`
16command into the binary, so patchelf is no help. Instead, it's very
17often easier to simply copy the desired files to their original store
18locations in initrd and not copy their entire runtime closure. This
19does mean that it is the burden of the developer to ensure that all
20necessary dependencies are copied in, as closures won't be
21consulted. However, it is rare that full closures are actually
22desirable, so in the traditional initrd, the developer was likely to
23do manual work on patching the dependencies explicitly anyway.
24
25# How it works
26
27This program is similar to its inspiration (`find-libs` from the
28traditional initrd), except that it also handles symlinks and
29directories according to certain rules. As input, it receives a
30sequence of pairs of paths. The first path is an object to copy into
31initrd. The second path (if not empty) is the path to a symlink that
32should be placed in the initrd, pointing to that object. How that
33object is copied depends on its type.
34
351. A regular file is copied directly to the same absolute path in the
36 initrd.
37
38 - If it is *also* an ELF file, then all of its direct shared
39 library dependencies are also listed as objects to be copied.
40
41 - If an unwrapped file exists as `.[filename]-wrapped`, then it is
42 also listed as an object to be copied.
43
442. A directory's direct children are listed as objects to be copied,
45 and a directory at the same absolute path in the initrd is created.
46
473. A symlink's target is listed as an object to be copied.
48
49There are a couple of quirks to mention here. First, the term "object"
50refers to the final file path that the developer intends to have
51copied into initrd. This means any parent directory is not considered
52an object just because its child was listed as an object in the
53program input; instead those intermediate directories are simply
54created in support of the target object. Second, shared libraries,
55directory children, and symlink targets aren't immediately recursed,
56because they simply get listed as objects themselves, and are
57therefore traversed when they themselves are processed. Finally,
58symlinks in the intermediate directories leading to an object are
59preserved, meaning an input object `/a/symlink/b` will just result in
60initrd containing `/a/symlink -> /target/b` and `/target/b`, even if
61`/target` has other children. Preserving symlinks in this manner is
62important for things like systemd.
63
64These rules automate the most important and obviously necessary
65copying that needs to be done in most cases, allowing programs and
66configuration files to go unpatched, while keeping the content of the
67initrd to a minimum.
68
69# Why Rust?
70
71- A prototype of this logic was written in Bash, in an attempt to keep
72 with its `find-libs` ancestor, but that program was difficult to
73 write, and ended up taking several minutes to run. This program runs
74 in less than a second, and the code is substantially easier to work
75 with.
76
77- This will not require end users to install a rust toolchain to use
78 NixOS, as long as this tool is cached by Hydra. And if you're
79 bootstrapping NixOS from source, rustc is already required anyway.
80
81- Rust was favored over Python for its type system, and because if you
82 want to go fast, why not go *really fast*?