···78---
910-Building this package twice does not produce the bit-by-bit identical result each time, making it harder to detect CI breaches. You can read more about this at https://reproducible-builds.org/ .
01112-Fixing bit-by-bit reproducibility also has additional advantages, such as avoiding hard-to-reproduce bugs, making content-addressed storage more effective and reducing rebuilds in such systems.
00000000000000000001314### Steps To Reproduce
15000000000016```
17-nix-build '<nixpkgs>' -A ... && nix-build '<nixpkgs>' -A ... --check --keep-failed
18```
1920-If this command completes successfully, no differences where found. However, when it ends in `error: derivation '<X>' may not be deterministic: output '<Y>' differs from '<Z>'`, you can use `diffoscope <Y> <Z>` to analyze the differences in the output of the two builds.
00002122-To view the build log of the build that produced the artifact in the binary cache:
000000002324```
25-nix-store --read-log $(nix-instantiate '<nixpkgs>' -A ...)
0000000000000026```
2728### Additional context
2930-(please share the relevant fragment of the diffoscope output here,
31-and any additional analysis you may have done)
···78---
910+<!--
11+Hello dear reporter,
1213+Thank you for bringing attention to this issue. Your insights are valuable to
14+us, and we appreciate the time you took to document the problem.
15+16+I wanted to kindly point out that in this issue template, it would be beneficial
17+to replace the placeholder `<package>` with the actual, canonical name of the
18+package you're reporting the issue for. Doing so will provide better context and
19+facilitate quicker troubleshooting for anyone who reads this issue in the
20+future.
21+22+Best regards
23+-->
24+25+Building this package multiple times does not yield bit-by-bit identical
26+results, complicating the detection of Continuous Integration (CI) breaches. For
27+more information on this issue, visit
28+[reproducible-builds.org](https://reproducible-builds.org/).
29+30+Fixing bit-by-bit reproducibility also has additional advantages, such as
31+avoiding hard-to-reproduce bugs, making content-addressed storage more effective
32+and reducing rebuilds in such systems.
3334### Steps To Reproduce
3536+In the following steps, replace `<package>` with the canonical name of the
37+package.
38+39+#### 1. Build the package
40+41+This step will build the package. Specific arguments are passed to the command
42+to keep the build artifacts so we can compare them in case of differences.
43+44+Execute the following command:
45+46```
47+nix-build '<nixpkgs>' -A <package> && nix-build '<nixpkgs>' -A <package> --check --keep-failed
48```
4950+Or using the new command line style:
51+52+```
53+nix build nixpkgs#<package> && nix build nixpkgs#<package> --rebuild --keep-failed
54+```
5556+#### 2. Compare the build artifacts
57+58+If the previous command completes successfully, no differences were found and
59+there's nothing to do, builds are reproducible.
60+If it terminates with the error message `error: derivation '<X>' may not be
61+deterministic: output '<Y>' differs from '<Z>'`, use `diffoscope` to investigate
62+the discrepancies between the two build outputs. You may need to add the
63+`--exclude-directory-metadata recursive` option to ignore files and directories
64+metadata (*e.g. timestamp*) differences.
6566```
67+nix run nixpkgs#diffoscopeMinimal -- --exclude-directory-metadata recursive <Y> <Z>
68+```
69+70+#### 3. Examine the build log
71+72+To examine the build log, use:
73+74+```
75+nix-store --read-log $(nix-instantiate '<nixpkgs>' -A <package>)
76+```
77+78+Or with the new command line style:
79+80+```
81+nix log $(nix path-info --derivation nixpkgs#<package>)
82```
8384### Additional context
8586+(please share the relevant fragment of the diffoscope output here, and any
87+additional analysis you may have done)
···162 getExe' pkgs.imagemagick "convert"
163 => "/nix/store/5rs48jamq7k6sal98ymj9l4k2bnwq515-imagemagick-7.1.1-15/bin/convert"
164 */
165+ getExe' = x: y:
166+ assert lib.assertMsg (lib.isDerivation x)
167+ "lib.meta.getExe': The first argument is of type ${builtins.typeOf x}, but it should be a derivation instead.";
168+ assert lib.assertMsg (lib.isString y)
169+ "lib.meta.getExe': The second argument is of type ${builtins.typeOf y}, but it should be a string instead.";
170+ assert lib.assertMsg (builtins.length (lib.splitString "/" y) == 1)
171+ "lib.meta.getExe': The second argument \"${y}\" is a nested path with a \"/\" character, but it should just be the name of the executable instead.";
172+ "${lib.getBin x}/bin/${y}";
173}
+28
lib/tests/misc.nix
···1906 expr = (with types; either int (listOf (either bool str))).description;
1907 expected = "signed integer or list of (boolean or string)";
1908 };
00000000000000000000000000001909}
···1906 expr = (with types; either int (listOf (either bool str))).description;
1907 expected = "signed integer or list of (boolean or string)";
1908 };
1909+1910+# Meta
1911+ testGetExe'Output = {
1912+ expr = getExe' {
1913+ type = "derivation";
1914+ out = "somelonghash";
1915+ bin = "somelonghash";
1916+ } "executable";
1917+ expected = "somelonghash/bin/executable";
1918+ };
1919+1920+ testGetExeOutput = {
1921+ expr = getExe {
1922+ type = "derivation";
1923+ out = "somelonghash";
1924+ bin = "somelonghash";
1925+ meta.mainProgram = "mainProgram";
1926+ };
1927+ expected = "somelonghash/bin/mainProgram";
1928+ };
1929+1930+ testGetExe'FailureFirstArg = testingThrow (
1931+ getExe' "not a derivation" "executable"
1932+ );
1933+1934+ testGetExe'FailureSecondArg = testingThrow (
1935+ getExe' { type = "derivation"; } "dir/executable"
1936+ );
1937}
···38 true`. This is generally safe behavior, but for anyone needing to opt out from
39 the check `users.users.${USERNAME}.ignoreShellProgramCheck = true` will do the job.
400041## New Services {#sec-release-23.11-new-services}
4243- [MCHPRS](https://github.com/MCHPR/MCHPRS), a multithreaded Minecraft server built for redstone. Available as [services.mchprs](#opt-services.mchprs.enable).
···351352- `service.borgmatic.settings.location` and `services.borgmatic.configurations.<name>.location` are deprecated, please move your options out of sections to the global scope.
35300354- `dagger` was removed because using a package called `dagger` and packaging it from source violates their trademark policy.
355356- `win-virtio` package was renamed to `virtio-win` to be consistent with the upstream package name.
···507- `ffmpeg` default upgraded from `ffmpeg_5` to `ffmpeg_6`.
508509- `fusuma` now enables the following plugins: [appmatcher](https://github.com/iberianpig/fusuma-plugin-appmatcher), [keypress](https://github.com/iberianpig/fusuma-plugin-keypress), [sendkey](https://github.com/iberianpig/fusuma-plugin-sendkey), [tap](https://github.com/iberianpig/fusuma-plugin-tap) and [wmctrl](https://github.com/iberianpig/fusuma-plugin-wmctrl).
00510511## Nixpkgs internals {#sec-release-23.11-nixpkgs-internals}
512
···38 true`. This is generally safe behavior, but for anyone needing to opt out from
39 the check `users.users.${USERNAME}.ignoreShellProgramCheck = true` will do the job.
4041+- Cassandra now defaults to 4.x, updated from 3.11.x.
42+43## New Services {#sec-release-23.11-new-services}
4445- [MCHPRS](https://github.com/MCHPR/MCHPRS), a multithreaded Minecraft server built for redstone. Available as [services.mchprs](#opt-services.mchprs.enable).
···353354- `service.borgmatic.settings.location` and `services.borgmatic.configurations.<name>.location` are deprecated, please move your options out of sections to the global scope.
355356+- `privacyidea` (and the corresponding `privacyidea-ldap-proxy`) has been removed from nixpkgs because it has severely outdated dependencies that became unmaintainable with nixpkgs' python package-set.
357+358- `dagger` was removed because using a package called `dagger` and packaging it from source violates their trademark policy.
359360- `win-virtio` package was renamed to `virtio-win` to be consistent with the upstream package name.
···511- `ffmpeg` default upgraded from `ffmpeg_5` to `ffmpeg_6`.
512513- `fusuma` now enables the following plugins: [appmatcher](https://github.com/iberianpig/fusuma-plugin-appmatcher), [keypress](https://github.com/iberianpig/fusuma-plugin-keypress), [sendkey](https://github.com/iberianpig/fusuma-plugin-sendkey), [tap](https://github.com/iberianpig/fusuma-plugin-tap) and [wmctrl](https://github.com/iberianpig/fusuma-plugin-wmctrl).
514+515+- `services.bitcoind` now properly respects the `enable` option.
516517## Nixpkgs internals {#sec-release-23.11-nixpkgs-internals}
518
···1-{ lib, fetchurl, version, astring, base, camlp-streams, cmdliner_1_0
2, cmdliner_1_1, csexp, dune-build-info, either, fix, fpath, menhirLib, menhirSdk
3-, ocaml-version, ocp-indent, odoc-parser, result, stdio, uuseg, uutf }:
45# The ocamlformat package have been split into two in version 0.25.1:
6# one for the library and one for the executable.
···23 "0.24.1" = "sha256-AjQl6YGPgOpQU3sjcaSnZsFJqZV9BYB+iKAE0tX0Qc4=";
24 "0.25.1" = "sha256-3I8qMwyjkws2yssmI7s2Dti99uSorNKT29niJBpv0z0=";
25 "0.26.0" = "sha256-AxSUq3cM7xCo9qocvrVmDkbDqmwM1FexEP7IWadeh30=";
026 }."${version}";
27 };
002829 odoc-parser_v = odoc-parser.override {
30 version = if lib.versionAtLeast version "0.24.0" then
···1+{ lib, fetchurl, version ? "0.26.1", astring, base, camlp-streams, cmdliner_1_0
2, cmdliner_1_1, csexp, dune-build-info, either, fix, fpath, menhirLib, menhirSdk
3+, ocaml-version, ocp-indent, odoc-parser, result, stdio, uuseg, uutf, ... }:
45# The ocamlformat package have been split into two in version 0.25.1:
6# one for the library and one for the executable.
···23 "0.24.1" = "sha256-AjQl6YGPgOpQU3sjcaSnZsFJqZV9BYB+iKAE0tX0Qc4=";
24 "0.25.1" = "sha256-3I8qMwyjkws2yssmI7s2Dti99uSorNKT29niJBpv0z0=";
25 "0.26.0" = "sha256-AxSUq3cM7xCo9qocvrVmDkbDqmwM1FexEP7IWadeh30=";
26+ "0.26.1" = "sha256-2gBuQn8VuexhL7gI1EZZm9m3w+4lq+s9VVdHpw10xtc=";
27 }."${version}";
28 };
29+30+ inherit version;
3132 odoc-parser_v = odoc-parser.override {
33 version = if lib.versionAtLeast version "0.24.0" then
···1+# Version can be selected with the 'version' argument, see generic.nix.
2+{ lib, callPackage, buildDunePackage, menhir, ... }@args:
34+let inherit (callPackage ./generic.nix args) src version library_deps;
56in assert (lib.versionAtLeast version "0.25.1");
7
···1+# Version can be selected with the 'version' argument, see generic.nix.
2{ lib
3, callPackage
4, buildDunePackage
···6, re
7, ocamlformat-lib
8, menhir
9+, ...
10+}@args:
1112+let inherit (callPackage ./generic.nix args) src version library_deps;
13in
1415lib.throwIf (lib.versionAtLeast ocaml.version "5.0" && !lib.versionAtLeast version "0.23")
···84 "-e"
85 (builtins.toFile "bash-builder.sh" ''
86 export CONFIG_SHELL=$SHELL
87+88+ # Normalize the NIX_BUILD_CORES variable. The value might be 0, which
89+ # means that we're supposed to try and auto-detect the number of
90+ # available CPU cores at run-time. We don't have nproc to detect the
91+ # number of available CPU cores so default to 1 if not set.
92+ NIX_BUILD_CORES="''${NIX_BUILD_CORES:-1}"
93+ if [ $NIX_BUILD_CORES -le 0 ]; then
94+ NIX_BUILD_CORES=1
95+ fi
96+ export NIX_BUILD_CORES
97+98 bash -eux $buildCommandPath
99 '')
100 ];
···54 "-e"
55 (builtins.toFile "bash-builder.sh" ''
56 export CONFIG_SHELL=$SHELL
57+58+ # Normalize the NIX_BUILD_CORES variable. The value might be 0, which
59+ # means that we're supposed to try and auto-detect the number of
60+ # available CPU cores at run-time.
61+ NIX_BUILD_CORES="''${NIX_BUILD_CORES:-1}"
62+ if ((NIX_BUILD_CORES <= 0)); then
63+ guess=$(nproc 2>/dev/null || true)
64+ ((NIX_BUILD_CORES = guess <= 0 ? 1 : guess))
65+ fi
66+ export NIX_BUILD_CORES
67+68 bash -eux $buildCommandPath
69 '')
70 ];
···711 pinentry_qt = throw "'pinentry_qt' has been renamed to/replaced by 'pinentry-qt'"; # Converted to throw 2023-09-10
712 pinentry_qt5 = pinentry-qt; # Added 2020-02-11
713 poetry2nix = throw "poetry2nix is now maintained out-of-tree. Please use https://github.com/nix-community/poetry2nix/"; # Added 2023-10-26
0714 probe-rs-cli = throw "probe-rs-cli is now part of the probe-rs package"; # Added 2023-07-03
715 processing3 = throw "'processing3' has been renamed to/replaced by 'processing'"; # Converted to throw 2023-09-10
716 prometheus-dmarc-exporter = dmarc-metrics-exporter; # added 2022-05-31
···711 pinentry_qt = throw "'pinentry_qt' has been renamed to/replaced by 'pinentry-qt'"; # Converted to throw 2023-09-10
712 pinentry_qt5 = pinentry-qt; # Added 2020-02-11
713 poetry2nix = throw "poetry2nix is now maintained out-of-tree. Please use https://github.com/nix-community/poetry2nix/"; # Added 2023-10-26
714+ privacyidea = throw "privacyidea has been removed from nixpkgs"; # Added 2023-10-31
715 probe-rs-cli = throw "probe-rs-cli is now part of the probe-rs package"; # Added 2023-07-03
716 processing3 = throw "'processing3' has been renamed to/replaced by 'processing'"; # Converted to throw 2023-09-10
717 prometheus-dmarc-exporter = dmarc-metrics-exporter; # added 2022-05-31