pyrox.dev
linux_5_2, linux_5_3: fixing nondeterminism
In 5.2 kernel a new mechanism was introduced which embeds the kernel
headers in the kernel image and exposes them in procfs for simplified
use by userland tools.
It was introduced in
https://github.com/torvalds/linux/commit/43d8ce9d65a54846d378545770991e65838981e0
and later modified a bit in
https://github.com/torvalds/linux/commit/f7b101d33046a837c2aa4526cef28a3c785d7af2
The archive containing the header files had nondeterminism through the
header files metadata - specifically `mtime`, but I also decided to
normalize some other aspects just in case.
In our default setup we currently compile this as a module, so to expose
the headers to test the functionality `kheaders` needs to be loaded.
See https://lkml.org/lkml/2019/10/4/1036 and
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=2cc99c9cdc8fde5e92e34f9655829449cebd3e00
I commented out the documentation part of the patch to make it cleanly apply to
5.2 and 5.3, see remark in the patch itself.
+86
pkgs/os-specific/linux/kernel/gen-kheaders-metadata.patch
+86
pkgs/os-specific/linux/kernel/gen-kheaders-metadata.patch
···
1
+
From 2cc99c9cdc8fde5e92e34f9655829449cebd3e00 Mon Sep 17 00:00:00 2001
2
+
From: Dmitry Goldin <dgoldin+lkml@protonmail.ch>
3
+
Date: Fri, 4 Oct 2019 10:40:07 +0000
4
+
Subject: kheaders: make headers archive reproducible
5
+
6
+
In commit 43d8ce9d65a5 ("Provide in-kernel headers to make
7
+
extending kernel easier") a new mechanism was introduced, for kernels
8
+
>=5.2, which embeds the kernel headers in the kernel image or a module
9
+
and exposes them in procfs for use by userland tools.
10
+
11
+
The archive containing the header files has nondeterminism caused by
12
+
header files metadata. This patch normalizes the metadata and utilizes
13
+
KBUILD_BUILD_TIMESTAMP if provided and otherwise falls back to the
14
+
default behaviour.
15
+
16
+
In commit f7b101d33046 ("kheaders: Move from proc to sysfs") it was
17
+
modified to use sysfs and the script for generation of the archive was
18
+
renamed to what is being patched.
19
+
20
+
Signed-off-by: Dmitry Goldin <dgoldin+lkml@protonmail.ch>
21
+
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
22
+
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
23
+
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
24
+
25
+
---
26
+
27
+
nixos note: This patch is from
28
+
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=2cc99c9cdc8fde5e92e34f9655829449cebd3e00
29
+
I commented out the documentation part here, so that it easily applies
30
+
to linux 5.2 and 5.3, which does not ship with the reproducible build
31
+
documentation yet, which only was introduced recently.
32
+
33
+
---
34
+
Documentation/kbuild/reproducible-builds.rst | 13 +++++++++----
35
+
kernel/gen_kheaders.sh | 5 ++++-
36
+
2 files changed, 13 insertions(+), 5 deletions(-)
37
+
38
+
#diff --git a/Documentation/kbuild/reproducible-builds.rst b/Documentation/kbuild/reproducible-builds.rst
39
+
#index ab92e98c89c8..503393854e2e 100644
40
+
# --- a/Documentation/kbuild/reproducible-builds.rst
41
+
#+++ b/Documentation/kbuild/reproducible-builds.rst
42
+
#@@ -16,16 +16,21 @@ the kernel may be unreproducible, and how to avoid them.
43
+
# Timestamps
44
+
# ----------
45
+
#
46
+
#-The kernel embeds a timestamp in two places:
47
+
#+The kernel embeds timestamps in three places:
48
+
#
49
+
# * The version string exposed by ``uname()`` and included in
50
+
# ``/proc/version``
51
+
#
52
+
# * File timestamps in the embedded initramfs
53
+
#
54
+
#-By default the timestamp is the current time. This must be overridden
55
+
#-using the `KBUILD_BUILD_TIMESTAMP`_ variable. If you are building
56
+
#-from a git commit, you could use its commit date.
57
+
#+* If enabled via ``CONFIG_IKHEADERS``, file timestamps of kernel
58
+
#+ headers embedded in the kernel or respective module,
59
+
#+ exposed via ``/sys/kernel/kheaders.tar.xz``
60
+
#+
61
+
#+By default the timestamp is the current time and in the case of
62
+
#+``kheaders`` the various files' modification times. This must
63
+
#+be overridden using the `KBUILD_BUILD_TIMESTAMP`_ variable.
64
+
#+If you are building from a git commit, you could use its commit date.
65
+
#
66
+
# The kernel does *not* use the ``__DATE__`` and ``__TIME__`` macros,
67
+
# and enables warnings if they are used. If you incorporate external
68
+
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
69
+
index 9ff449888d9c..aff79e461fc9 100755
70
+
--- a/kernel/gen_kheaders.sh
71
+
+++ b/kernel/gen_kheaders.sh
72
+
@@ -71,7 +71,10 @@ done | cpio --quiet -pd $cpio_dir >/dev/null 2>&1
73
+
find $cpio_dir -type f -print0 |
74
+
xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
75
+
76
+
-tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null
77
+
+# Create archive and try to normalize metadata for reproducibility
78
+
+tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
79
+
+ --owner=0 --group=0 --sort=name --numeric-owner \
80
+
+ -Jcf $tarfile -C $cpio_dir/ . > /dev/null
81
+
82
+
echo "$src_files_md5" > kernel/kheaders.md5
83
+
echo "$obj_files_md5" >> kernel/kheaders.md5
84
+
--
85
+
cgit 1.2-0.3.lf.el7
86
+
+3
-1
pkgs/os-specific/linux/kernel/manual-config.nix
+3
-1
pkgs/os-specific/linux/kernel/manual-config.nix
···
94
94
patches =
95
95
map (p: p.patch) kernelPatches
96
96
# Required for deterministic builds along with some postPatch magic.
97
-
++ optional (stdenv.lib.versionAtLeast version "4.13") ./randstruct-provide-seed.patch;
97
+
++ optional (stdenv.lib.versionAtLeast version "4.13") ./randstruct-provide-seed.patch
98
+
# Fixes determinism by normalizing metadata for the archive of kheaders
99
+
++ optional (stdenv.lib.versionAtLeast version "5.2") ./gen-kheaders-metadata.patch;
98
100
99
101
prePatch = ''
100
102
for mf in $(find -name Makefile -o -name Makefile.include -o -name install.sh); do