···11+# Checks pkgs/by-name (see pkgs/by-name/README.md)
22+# using the nixpkgs-check-by-name tool (see pkgs/test/nixpkgs-check-by-name)
33+name: Check pkgs/by-name
44+55+# The pre-built tool is fetched from a channel,
66+# making it work predictable on all PRs
77+on: pull_request
88+99+# The tool doesn't need any permissions, it only outputs success or not based on the checkout
1010+permissions: {}
1111+1212+jobs:
1313+ check:
1414+ # This is x86_64-linux, for which the tool is always prebuilt on the nixos-* channels,
1515+ # as specified in nixos/release-combined.nix
1616+ runs-on: ubuntu-latest
1717+ steps:
1818+ - uses: actions/checkout@v3
1919+ - uses: cachix/install-nix-action@v22
2020+ - name: Determining channel to use for dependencies
2121+ run: |
2222+ echo "Determining which channel to use for PR base branch $GITHUB_BASE_REF"
2323+ if [[ "$GITHUB_BASE_REF" =~ ^(release|staging|staging-next)-([0-9][0-9]\.[0-9][0-9])$ ]]; then
2424+ # Use the release channel for all PRs to release-XX.YY, staging-XX.YY and staging-next-XX.YY
2525+ channel=nixos-${BASH_REMATCH[2]}
2626+ echo "PR is for a release branch, using release channel $channel"
2727+ else
2828+ # Use the nixos-unstable channel for all other PRs
2929+ channel=nixos-unstable
3030+ echo "PR is for a non-release branch, using unstable channel $channel"
3131+ fi
3232+ echo "channel=$channel" >> "$GITHUB_ENV"
3333+ - name: Fetching latest version of channel
3434+ run: |
3535+ echo "Fetching latest version of channel $channel"
3636+ # This is probably the easiest way to get Nix to output the path to a downloaded channel!
3737+ nixpkgs=$(nix-instantiate --find-file nixpkgs -I nixpkgs=channel:"$channel")
3838+ # This file only exists in channels
3939+ rev=$(<"$nixpkgs"/.git-revision)
4040+ echo "Channel $channel is at revision $rev"
4141+ echo "nixpkgs=$nixpkgs" >> "$GITHUB_ENV"
4242+ echo "rev=$rev" >> "$GITHUB_ENV"
4343+ - name: Fetching pre-built nixpkgs-check-by-name from the channel
4444+ run: |
4545+ echo "Fetching pre-built nixpkgs-check-by-name from channel $channel at revision $rev"
4646+ # Passing --max-jobs 0 makes sure that we won't build anything
4747+ nix-build "$nixpkgs" -A tests.nixpkgs-check-by-name --max-jobs 0
4848+ - name: Running nixpkgs-check-by-name
4949+ run: result/bin/nixpkgs-check-by-name .
···4949 # Wait for lxd to settle
5050 machine.succeed("lxd waitready")
51515252+ # no preseed should mean no service
5353+ machine.fail("systemctl status lxd-preseed.service")
5454+5255 machine.succeed("lxd init --minimal")
53565457 machine.succeed(
+3-1
nixos/tests/lxd/default.nix
···22 system ? builtins.currentSystem,
33 config ? {},
44 pkgs ? import ../../.. {inherit system config;},
55+ handleTestOn,
56}: {
67 container = import ./container.nix {inherit system pkgs;};
78 nftables = import ./nftables.nix {inherit system pkgs;};
99+ preseed = import ./preseed.nix {inherit system pkgs;};
810 ui = import ./ui.nix {inherit system pkgs;};
99- virtual-machine = import ./virtual-machine.nix { inherit system pkgs; };
1111+ virtual-machine = handleTestOn ["x86_64-linux"] ./virtual-machine.nix { inherit system pkgs; };
1012}
···7788- [`top-level`](./top-level): Entrypoints, package set aggregations
99 - [`impure.nix`](./top-level/impure.nix), [`default.nix`](./top-level/default.nix), [`config.nix`](./top-level/config.nix): Definitions for the evaluation entry point of `import <nixpkgs>`
1010- - [`stage.nix`](./top-level/stage.nix), [`all-packages.nix`](./top-level/all-packages.nix), [`splice.nix`](./top-level/splice.nix): Definitions for the top-level attribute set made available through `import <nixpkgs> {…}`
1010+ - [`stage.nix`](./top-level/stage.nix), [`all-packages.nix`](./top-level/all-packages.nix), [`by-name-overlay.nix`](./top-level/by-name-overlay.nix), [`splice.nix`](./top-level/splice.nix): Definitions for the top-level attribute set made available through `import <nixpkgs> {…}`
1111 - `*-packages.nix`, [`linux-kernels.nix`](./top-level/linux-kernels.nix), [`unixtools.nix`](./top-level/unixtools.nix): Aggregations of nested package sets defined in `development`
1212 - [`aliases.nix`](./top-level/aliases.nix), [`python-aliases.nix`](./top-level/python-aliases.nix): Aliases for package definitions that have been renamed or removed
1313 - `release*.nix`, [`make-tarball.nix`](./top-level/make-tarball.nix), [`packages-config.nix`](./top-level/packages-config.nix), [`metrics.nix`](./top-level/metrics.nix), [`nixpkgs-basic-release-checks.nix`](./top-level/nixpkgs-basic-release-checks.nix): Entry-points and utilities used by Hydra for continuous integration
···1919- [`stdenv`](./stdenv): [Standard environment](https://nixos.org/manual/nixpkgs/stable/#part-stdenv)
2020- [`pkgs-lib`](./pkgs-lib): Definitions for utilities that need packages but are not needed for packages
2121- [`test`](./test): Tests not directly associated with any specific packages
2222+- [`by-name`](./by-name): Top-level packages organised by name ([docs](./by-name/README.md))
2223- All other directories loosely categorise top-level packages definitions, see [category hierarchy][categories]
23242425## Quick Start to Adding a Package
···4950 $ cd nixpkgs
5051 ```
51525252-2. Find a good place in the Nixpkgs tree to add the Nix expression for your package. For instance, a library package typically goes into `pkgs/development/libraries/pkgname`, while a web browser goes into `pkgs/applications/networking/browsers/pkgname`. See the [category hierarchy section][categories] for some hints on the tree organisation. Create a directory for your package, e.g.
5353+2. Create a package directory `pkgs/by-name/so/some-package` where `some-package` is the package name and `so` is the lowercased 2-letter prefix of the package name:
53545455 ```ShellSession
5555- $ mkdir pkgs/development/libraries/libfoo
5656+ $ mkdir -p pkgs/by-name/so/some-package
5657 ```
57585858-3. In the package directory, create a Nix expression — a piece of code that describes how to build the package. In this case, it should be a _function_ that is called with the package dependencies as arguments, and returns a build of the package in the Nix store. The expression should usually be called `default.nix`.
5959+ For more detailed information, see [here](./by-name/README.md).
6060+6161+3. Create a `package.nix` file in the package directory, containing a Nix expression — a piece of code that describes how to build the package. In this case, it should be a _function_ that is called with the package dependencies as arguments, and returns a build of the package in the Nix store.
59626063 ```ShellSession
6161- $ emacs pkgs/development/libraries/libfoo/default.nix
6262- $ git add pkgs/development/libraries/libfoo/default.nix
6464+ $ emacs pkgs/by-name/so/some-package/package.nix
6565+ $ git add pkgs/by-name/so/some-package/package.nix
6366 ```
64676565- You can have a look at the existing Nix expressions under `pkgs/` to see how it’s done. Here are some good ones:
6868+ You can have a look at the existing Nix expressions under `pkgs/` to see how it’s done, some of which are also using the [category hierarchy](#category-hierarchy).
6969+ Here are some good ones:
66706767- - GNU Hello: [`pkgs/applications/misc/hello/default.nix`](applications/misc/hello/default.nix). Trivial package, which specifies some `meta` attributes which is good practice.
7171+ - GNU Hello: [`pkgs/by-name/he/hello/package.nix`](./by-name/he/hello/package.nix). Trivial package, which specifies some `meta` attributes which is good practice.
68726973 - GNU cpio: [`pkgs/tools/archivers/cpio/default.nix`](tools/archivers/cpio/default.nix). Also a simple package. The generic builder in `stdenv` does everything for you. It has no dependencies beyond `stdenv`.
7074···94989599 The exact syntax and semantics of the Nix expression language, including the built-in function, are [described in the Nix manual](https://nixos.org/manual/nix/stable/language/).
961009797-4. Add a call to the function defined in the previous step to [`pkgs/top-level/all-packages.nix`](top-level/all-packages.nix) with some descriptive name for the variable, e.g. `libfoo`.
9898-9999- ```ShellSession
100100- $ emacs pkgs/top-level/all-packages.nix
101101- ```
102102-103103- The attributes in that file are sorted by category (like “Development / Libraries”) that more-or-less correspond to the directory structure of Nixpkgs, and then by attribute name.
104104-1051015. To test whether the package builds, run the following command from the root of the nixpkgs source tree:
106102107103 ```ShellSession
108108- $ nix-build -A libfoo
104104+ $ nix-build -A some-package
109105 ```
110106111111- where `libfoo` should be the variable name defined in the previous step. You may want to add the flag `-K` to keep the temporary build directory in case something fails. If the build succeeds, a symlink `./result` to the package in the Nix store is created.
107107+ where `some-package` should be the package name. You may want to add the flag `-K` to keep the temporary build directory in case something fails. If the build succeeds, a symlink `./result` to the package in the Nix store is created.
1121081131096. If you want to install the package into your profile (optional), do
114110···121117## Category Hierarchy
122118[categories]: #category-hierarchy
123119124124-Each package should be stored in its own directory somewhere in the `pkgs/` tree, i.e. in `pkgs/category/subcategory/.../pkgname`. Below are some rules for picking the right category for a package. Many packages fall under several categories; what matters is the _primary_ purpose of a package. For example, the `libxml2` package builds both a library and some tools; but it’s a library foremost, so it goes under `pkgs/development/libraries`.
120120+Most top-level packages are organised in a loosely-categorised directory hierarchy in this directory.
121121+See the [overview](#overview) for which directories are part of this.
122122+123123+This category hierarchy is partially deprecated and will be migrated away over time.
124124+The new `pkgs/by-name` directory ([docs](./by-name/README.md)) should be preferred instead.
125125+The category hierarchy may still be used for packages that should be imported using an alternate `callPackage`, such as `python3Packages.callPackage` or `libsForQt5.callPackage`.
126126+127127+If that is the case for a new package, here are some rules for picking the right category.
128128+Many packages fall under several categories; what matters is the _primary_ purpose of a package.
129129+For example, the `libxml2` package builds both a library and some tools; but it’s a library foremost, so it goes under `pkgs/development/libraries`.
125130126126-When in doubt, consider refactoring the `pkgs/` tree, e.g. creating new categories or splitting up an existing category.
131131+<details>
132132+<summary>Categories</summary>
127133128134**If it’s used to support _software development_:**
129135···298304**Else:**
299305300306- `misc`
307307+308308+</details>
301309302310# Conventions
303311
···11+# Name-based package directories
22+33+The structure of this directory maps almost directly to top-level package attributes.
44+This is the recommended way to add new top-level packages to Nixpkgs [when possible](#limitations).
55+66+## Example
77+88+The top-level package `pkgs.some-package` may be declared by setting up this file structure:
99+1010+```
1111+pkgs
1212+└── by-name
1313+ ├── so
1414+ ┊ ├── some-package
1515+ ┊ └── package.nix
1616+1717+```
1818+1919+Where `some-package` is the package name and `so` is the lowercased 2-letter prefix of the package name.
2020+2121+The `package.nix` may look like this:
2222+2323+```nix
2424+# A function taking an attribute set as an argument
2525+{
2626+ # Get access to top-level attributes for use as dependencies
2727+ lib,
2828+ stdenv,
2929+ libbar,
3030+3131+ # Make this derivation configurable using `.override { enableBar = true }`
3232+ enableBar ? false,
3333+}:
3434+3535+# The return value must be a derivation
3636+stdenv.mkDerivation {
3737+ # ...
3838+ buildInputs =
3939+ lib.optional enableBar libbar;
4040+}
4141+```
4242+4343+You can also split up the package definition into more files in the same directory if necessary.
4444+4545+Once defined, the package can be built from the Nixpkgs root directory using:
4646+```
4747+nix-build -A some-package
4848+```
4949+5050+See the [general package conventions](../README.md#conventions) for more information on package definitions.
5151+5252+### Changing implicit attribute defaults
5353+5454+The above expression is called using these arguments by default:
5555+```nix
5656+{
5757+ lib = pkgs.lib;
5858+ stdenv = pkgs.stdenv;
5959+ libbar = pkgs.libbar;
6060+}
6161+```
6262+6363+But the package might need `pkgs.libbar_2` instead.
6464+While the function could be changed to take `libbar_2` directly as an argument,
6565+this would change the `.override` interface, breaking code like `.override { libbar = ...; }`.
6666+So instead it is preferable to use the same generic parameter name `libbar`
6767+and override its value in [`pkgs/top-level/all-packages.nix`](../top-level/all-packages.nix):
6868+6969+```nix
7070+libfoo = callPackage ../by-name/so/somePackage/package.nix {
7171+ libbar = libbar_2;
7272+};
7373+```
7474+7575+## Limitations
7676+7777+There's some limitations as to which packages can be defined using this structure:
7878+7979+- Only packages defined using `pkgs.callPackage`.
8080+ This excludes packages defined using `pkgs.python3Packages.callPackage ...`.
8181+8282+ Instead use the [category hierarchy](../README.md#category-hierarchy) for such attributes.
8383+8484+- Only top-level packages.
8585+ This excludes packages for other package sets like `pkgs.pythonPackages.*`.
8686+8787+ Refer to the definition and documentation of the respective package set to figure out how such packages can be declared.
8888+8989+## Validation
9090+9191+CI performs [certain checks](../test/nixpkgs-check-by-name/README.md#validity-checks) on the `pkgs/by-name` structure.
9292+This is done using the [`nixpkgs-check-by-name` tool](../test/nixpkgs-check-by-name).
9393+The version of this tool used is the one that corresponds to the NixOS channel of the PR base branch.
9494+See [here](../../.github/workflows/check-by-name.yml) for details.
9595+9696+The tool can be run locally using
9797+9898+```bash
9999+nix-build -A tests.nixpkgs-check-by-name
100100+result/bin/nixpkgs-check-by-name .
101101+```
+2-2
pkgs/data/fonts/sarasa-gothic/default.nix
···2233stdenvNoCC.mkDerivation rec {
44 pname = "sarasa-gothic";
55- version = "0.41.6";
55+ version = "0.41.8";
6677 src = fetchurl {
88 # Use the 'ttc' files here for a smaller closure size.
99 # (Using 'ttf' files gives a closure size about 15x larger, as of November 2021.)
1010 url = "https://github.com/be5invis/Sarasa-Gothic/releases/download/v${version}/sarasa-gothic-ttc-${version}.7z";
1111- hash = "sha256-6CDK9DNjBQ5EPp562na0DOWFmlxnlVl8Z8pwm3pGQ9A=";
1111+ hash = "sha256-TLEyCFQfBsCCHqnYmJNMJFkM+iKWqly3969zw9AbTFQ=";
1212 };
13131414 sourceRoot = ".";
···11# Nixpkgs pkgs/by-name checker
2233This directory implements a program to check the [validity](#validity-checks) of the `pkgs/by-name` Nixpkgs directory once introduced.
44+It is being used by [this GitHub Actions workflow](../../../.github/workflows/check-by-name.yml).
45This is part of the implementation of [RFC 140](https://github.com/NixOS/rfcs/pull/140).
5667## API
7888-This API may be changed over time if the CI making use of it is adjusted to deal with the change appropriately, see [Hydra builds](#hydra-builds).
99+This API may be changed over time if the CI workflow making use of it is adjusted to deal with the change appropriately.
9101011- Command line: `nixpkgs-check-by-name <NIXPKGS>`
1112- Arguments:
···11+# This file turns the pkgs/by-name directory (see its README.md for more info) into an overlay that adds all the defined packages.
22+# No validity checks are done here,
33+# instead this file is optimised for performance,
44+# and validity checks are done by CI on PRs.
55+66+# Type: Path -> Overlay
77+baseDirectory:
88+let
99+ # Because of Nix's import-value cache, importing lib is free
1010+ lib = import ../../lib;
1111+1212+ inherit (builtins)
1313+ readDir
1414+ ;
1515+1616+ inherit (lib.attrsets)
1717+ mapAttrs
1818+ mapAttrsToList
1919+ mergeAttrsList
2020+ ;
2121+2222+ # Package files for a single shard
2323+ # Type: String -> String -> AttrsOf Path
2424+ namesForShard = shard: type:
2525+ if type != "directory" then
2626+ # Ignore all non-directories. Technically only README.md is allowed as a file in the base directory, so we could alternatively:
2727+ # - Assume that README.md is the only file and change the condition to `shard == "README.md"` for a minor performance improvement.
2828+ # This would however cause very poor error messages if there's other files.
2929+ # - Ensure that README.md is the only file, throwing a better error message if that's not the case.
3030+ # However this would make for a poor code architecture, because one type of error would have to be duplicated in the validity checks and here.
3131+ # Additionally in either of those alternatives, we would have to duplicate the hardcoding of "README.md"
3232+ { }
3333+ else
3434+ mapAttrs
3535+ (name: _: baseDirectory + "/${shard}/${name}/package.nix")
3636+ (readDir (baseDirectory + "/${shard}"));
3737+3838+ # The attribute set mapping names to the package files defining them
3939+ # This is defined up here in order to allow reuse of the value (it's kind of expensive to compute)
4040+ # if the overlay has to be applied multiple times
4141+ packageFiles = mergeAttrsList (mapAttrsToList namesForShard (readDir baseDirectory));
4242+in
4343+# TODO: Consider optimising this using `builtins.deepSeq packageFiles`,
4444+# which could free up the above thunks and reduce GC times.
4545+# Currently this would be hard to measure until we have more packages
4646+# and ideally https://github.com/NixOS/nix/pull/8895
4747+self: super:
4848+mapAttrs (name: file:
4949+ self.callPackage file { }
5050+) packageFiles
+8
pkgs/top-level/stage.nix
···88 arguments. Normal users should not import this directly but instead
99 import `pkgs/default.nix` or `default.nix`. */
10101111+let
1212+ # An overlay to auto-call packages in ../by-name.
1313+ # By defining it at the top of the file,
1414+ # this value gets reused even if this file is imported multiple times,
1515+ # thanks to Nix's import-value cache.
1616+ autoCalledPackages = import ./by-name-overlay.nix ../by-name;
1717+in
11181219{ ## Misc parameters kept the same for all stages
1320 ##
···279286 stdenvAdapters
280287 trivialBuilders
281288 splice
289289+ autoCalledPackages
282290 allPackages
283291 otherPackageSets
284292 aliases