···11+22+# Developing the NixOS Test Driver {#chap-developing-the-test-driver}
33+44+The NixOS test framework is a project of its own.
55+66+It consists of roughly the following components:
77+88+ - `nixos/lib/test-driver`: The Python framework that sets up the test and runs the [`testScript`](#test-opt-testScript)
99+ - `nixos/lib/testing`: The Nix code responsible for the wiring, written using the (NixOS) Module System.
1010+1111+These components are exposed publicly through:
1212+1313+ - `nixos/lib/default.nix`: The public interface that exposes the `nixos/lib/testing` entrypoint.
1414+ - `flake.nix`: Exposes the `lib.nixos`, including the public test interface.
1515+1616+Beyond the test driver itself, its integration into NixOS and Nixpkgs is important.
1717+1818+ - `pkgs/top-level/all-packages.nix`: Defines the `nixosTests` attribute, used
1919+ by the package `tests` attributes and OfBorg.
2020+ - `nixos/release.nix`: Defines the `tests` attribute built by Hydra, independently, but analogous to `nixosTests`
2121+ - `nixos/release-combined.nix`: Defines which tests are channel blockers.
2222+2323+Finally, we have legacy entrypoints that users should move away from, but are cared for on a best effort basis.
2424+These include `pkgs.nixosTest`, `testing-python.nix` and `make-test-python.nix`.
2525+2626+## Testing changes to the test framework {#sec-test-the-test-framework}
2727+2828+When making significant changes to the test framework, we run the tests on Hydra, to avoid disrupting the larger NixOS project.
2929+3030+For this, we use the `python-test-refactoring` branch in the `NixOS/nixpkgs` repository, and its [corresponding Hydra jobset](https://hydra.nixos.org/jobset/nixos/python-test-refactoring).
3131+This branch is used as a pointer, and not as a feature branch.
3232+3333+1. Rebase the PR onto a recent, good evaluation of `nixos-unstable`
3434+2. Create a baseline evaluation by force-pushing this revision of `nixos-unstable` to `python-test-refactoring`.
3535+3. Note the evaluation number (we'll call it `<previous>`)
3636+4. Push the PR to `python-test-refactoring` and evaluate the PR on Hydra
3737+5. Create a comparison URL by navigating to the latest build of the PR and adding to the URL `?compare=<previous>`. This is not necessary for the evaluation that comes right after the baseline.
3838+3939+Review the removed tests and newly failed tests using the constructed URL; otherwise you will accidentally compare iterations of the PR instead of changes to the PR base.
4040+4141+As we currently have some flaky tests, newly failing tests are expected, but should be reviewed to make sure that
4242+ - The number of failures did not increase significantly.
4343+ - All failures that do occur can reasonably be assumed to fail for a different reason than the changes.
···1515 description = "Quickly get ANSI escape sequences";
1616 longDescription = ''
1717 CLI utility called "ansi" to quickly get ANSI escape sequences. Supports
1818- the colors and styles, such as bold or italic.";
1818+ the colors and styles, such as bold or italic.
1919 '';
2020 homepage = "https://github.com/phip1611/ansi-escape-sequences-cli";
2121 license = with licenses; [ mit ];
+3-3
pkgs/servers/plex/raw.nix
···1212# server, and the FHS userenv and corresponding NixOS module should
1313# automatically pick up the changes.
1414stdenv.mkDerivation rec {
1515- version = "1.30.2.6563-3d4dc0cce";
1515+ version = "1.31.0.6654-02189b09f";
1616 pname = "plexmediaserver";
17171818 # Fetch the source
1919 src = if stdenv.hostPlatform.system == "aarch64-linux" then fetchurl {
2020 url = "https://downloads.plex.tv/plex-media-server-new/${version}/debian/plexmediaserver_${version}_arm64.deb";
2121- sha256 = "0sz6xc484flh1cnlrvwin7x34bl118yy2mwj034f8p9ngiy5hrkw";
2121+ sha256 = "sha256-ttkvYD+ALxfZpQutI1VyTbmQi/7hmvZ+YMUv3lskeWU=";
2222 } else fetchurl {
2323 url = "https://downloads.plex.tv/plex-media-server-new/${version}/debian/plexmediaserver_${version}_amd64.deb";
2424- sha256 = "0ymxfy3s9nygv9syiy2bdwmjfqg8m4i5n8c37z1ib6393iwj8mgi";
2424+ sha256 = "sha256-TTEcyIBFiuJTNHeJ9wu+4o2ol72oCvM9FdDPC83J3Mc=";
2525 };
26262727 outputs = [ "out" "basedb" ];