···1# Python {#python}
20000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003## User Guide {#user-guide}
45### Using Python {#using-python}
···993example we use `buildPythonPackage` that is part of the set `python3Packages`,
994and in this case the `python3` interpreter is automatically used.
995996-## Reference {#reference}
997-998-### Interpreters {#interpreters}
999-1000-| Package | Aliases | Interpreter |
1001-|------------|-----------------|-------------|
1002-| python27 | python2, python | CPython 2.7 |
1003-| python38 | | CPython 3.8 |
1004-| python39 | | CPython 3.9 |
1005-| python310 | python3 | CPython 3.10 |
1006-| python311 | | CPython 3.11 |
1007-| python312 | | CPython 3.12 |
1008-| pypy27 | pypy2, pypy | PyPy2.7 |
1009-| pypy39 | pypy3 | PyPy 3.9 |
1010-1011-The Nix expressions for the interpreters can be found in
1012-`pkgs/development/interpreters/python`.
1013-1014-All packages depending on any Python interpreter get appended
1015-`out/{python.sitePackages}` to `$PYTHONPATH` if such directory
1016-exists.
1017-1018-#### Missing `tkinter` module standard library {#missing-tkinter-module-standard-library}
1019-1020-To reduce closure size the `Tkinter`/`tkinter` is available as a separate package, `pythonPackages.tkinter`.
1021-1022-#### Attributes on interpreters packages {#attributes-on-interpreters-packages}
1023-1024-Each interpreter has the following attributes:
1025-1026-- `libPrefix`. Name of the folder in `${python}/lib/` for corresponding interpreter.
1027-- `interpreter`. Alias for `${python}/bin/${executable}`.
1028-- `buildEnv`. Function to build python interpreter environments with extra packages bundled together. See section *python.buildEnv function* for usage and documentation.
1029-- `withPackages`. Simpler interface to `buildEnv`. See section *python.withPackages function* for usage and documentation.
1030-- `sitePackages`. Alias for `lib/${libPrefix}/site-packages`.
1031-- `executable`. Name of the interpreter executable, e.g. `python3.10`.
1032-- `pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
1033-1034-### Optimizations {#optimizations}
1035-1036-The Python interpreters are by default not built with optimizations enabled, because
1037-the builds are in that case not reproducible. To enable optimizations, override the
1038-interpreter of interest, e.g using
1039-1040-```
1041-let
1042- pkgs = import ./. {};
1043- mypython = pkgs.python3.override {
1044- enableOptimizations = true;
1045- reproducibleBuild = false;
1046- self = mypython;
1047- };
1048-in mypython
1049-```
1050-1051-### Building packages and applications {#building-packages-and-applications}
1052-1053-Python libraries and applications that use `setuptools` or
1054-`distutils` are typically built with respectively the `buildPythonPackage` and
1055-`buildPythonApplication` functions. These two functions also support installing a `wheel`.
1056-1057-All Python packages reside in `pkgs/top-level/python-packages.nix` and all
1058-applications elsewhere. In case a package is used as both a library and an
1059-application, then the package should be in `pkgs/top-level/python-packages.nix`
1060-since only those packages are made available for all interpreter versions. The
1061-preferred location for library expressions is in
1062-`pkgs/development/python-modules`. It is important that these packages are
1063-called from `pkgs/top-level/python-packages.nix` and not elsewhere, to guarantee
1064-the right version of the package is built.
1065-1066-Based on the packages defined in `pkgs/top-level/python-packages.nix` an
1067-attribute set is created for each available Python interpreter. The available
1068-sets are
1069-1070-* `pkgs.python27Packages`
1071-* `pkgs.python3Packages`
1072-* `pkgs.python38Packages`
1073-* `pkgs.python39Packages`
1074-* `pkgs.python310Packages`
1075-* `pkgs.python311Packages`
1076-* `pkgs.pypyPackages`
1077-1078-and the aliases
1079-1080-* `pkgs.python2Packages` pointing to `pkgs.python27Packages`
1081-* `pkgs.python3Packages` pointing to `pkgs.python310Packages`
1082-* `pkgs.pythonPackages` pointing to `pkgs.python2Packages`
1083-1084-#### `buildPythonPackage` function {#buildpythonpackage-function}
1085-1086-The `buildPythonPackage` function is implemented in
1087-`pkgs/development/interpreters/python/mk-python-derivation.nix`
1088-using setup hooks.
1089-1090-The following is an example:
1091-1092-```nix
1093-{ lib
1094-, buildPythonPackage
1095-, fetchPypi
1096-1097-# build-system
1098-, setuptools-scm
1099-1100-# dependencies
1101-, attrs
1102-, pluggy
1103-, py
1104-, setuptools
1105-, six
1106-1107-# tests
1108-, hypothesis
1109- }:
1110-1111-buildPythonPackage rec {
1112- pname = "pytest";
1113- version = "3.3.1";
1114- format = "setuptools";
1115-1116- src = fetchPypi {
1117- inherit pname version;
1118- hash = "sha256-z4Q23FnYaVNG/NOrKW3kZCXsqwDWQJbOvnn7Ueyy65M=";
1119- };
1120-1121- postPatch = ''
1122- # don't test bash builtins
1123- rm testing/test_argcomplete.py
1124- '';
1125-1126- nativeBuildInputs = [
1127- setuptools-scm
1128- ];
1129-1130- propagatedBuildInputs = [
1131- attrs
1132- py
1133- setuptools
1134- six
1135- pluggy
1136- ];
1137-1138- nativeCheckInputs = [
1139- hypothesis
1140- ];
1141-1142- meta = with lib; {
1143- changelog = "https://github.com/pytest-dev/pytest/releases/tag/${version}";
1144- description = "Framework for writing tests";
1145- homepage = "https://github.com/pytest-dev/pytest";
1146- license = licenses.mit;
1147- maintainers = with maintainers; [ domenkozar lovek323 madjar lsix ];
1148- };
1149-}
1150-```
1151-1152-The `buildPythonPackage` mainly does four things:
1153-1154-* In the `buildPhase`, it calls `${python.pythonForBuild.interpreter} setup.py bdist_wheel` to
1155- build a wheel binary zipfile.
1156-* In the `installPhase`, it installs the wheel file using `pip install *.whl`.
1157-* In the `postFixup` phase, the `wrapPythonPrograms` bash function is called to
1158- wrap all programs in the `$out/bin/*` directory to include `$PATH`
1159- environment variable and add dependent libraries to script's `sys.path`.
1160-* In the `installCheck` phase, `${python.interpreter} setup.py test` is run.
1161-1162-By default tests are run because `doCheck = true`. Test dependencies, like
1163-e.g. the test runner, should be added to `nativeCheckInputs`.
1164-1165-By default `meta.platforms` is set to the same value
1166-as the interpreter unless overridden otherwise.
1167-1168-##### `buildPythonPackage` parameters {#buildpythonpackage-parameters}
1169-1170-All parameters from `stdenv.mkDerivation` function are still supported. The
1171-following are specific to `buildPythonPackage`:
1172-1173-* `catchConflicts ? true`: If `true`, abort package build if a package name
1174- appears more than once in dependency tree. Default is `true`.
1175-* `disabled ? false`: If `true`, package is not built for the particular Python
1176- interpreter version.
1177-* `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
1178-* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
1179- variable in wrapped programs.
1180-* `format ? "setuptools"`: Format of the source. Valid options are
1181- `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`.
1182- `"setuptools"` is for when the source has a `setup.py` and `setuptools` is
1183- used to build a wheel, `flit`, in case `flit` should be used to build a wheel,
1184- and `wheel` in case a wheel is provided. Use `other` when a custom
1185- `buildPhase` and/or `installPhase` is needed.
1186-* `makeWrapperArgs ? []`: A list of strings. Arguments to be passed to
1187- `makeWrapper`, which wraps generated binaries. By default, the arguments to
1188- `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling
1189- the binary. Additional arguments here can allow a developer to set environment
1190- variables which will be available when the binary is run. For example,
1191- `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
1192-* `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this
1193- defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications to `""`.
1194-* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip
1195- install`. To pass options to `python setup.py install`, use
1196- `--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`.
1197-* `pipBuildFlags ? []`: A list of strings. Arguments to be passed to `pip wheel`.
1198-* `pypaBuildFlags ? []`: A list of strings. Arguments to be passed to `python -m build --wheel`.
1199-* `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages
1200- in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`).
1201-* `preShellHook`: Hook to execute commands before `shellHook`.
1202-* `postShellHook`: Hook to execute commands after `shellHook`.
1203-* `removeBinByteCode ? true`: Remove bytecode from `/bin`. Bytecode is only
1204- created when the filenames end with `.py`.
1205-* `setupPyGlobalFlags ? []`: List of flags passed to `setup.py` command.
1206-* `setupPyBuildFlags ? []`: List of flags passed to `setup.py build_ext` command.
1207-1208-The `stdenv.mkDerivation` function accepts various parameters for describing
1209-build inputs (see "Specifying dependencies"). The following are of special
1210-interest for Python packages, either because these are primarily used, or
1211-because their behaviour is different:
1212-1213-* `nativeBuildInputs ? []`: Build-time only dependencies. Typically executables
1214- as well as the items listed in `setup_requires`.
1215-* `buildInputs ? []`: Build and/or run-time dependencies that need to be
1216- compiled for the host machine. Typically non-Python libraries which are being
1217- linked.
1218-* `nativeCheckInputs ? []`: Dependencies needed for running the `checkPhase`. These
1219- are added to `nativeBuildInputs` when `doCheck = true`. Items listed in
1220- `tests_require` go here.
1221-* `propagatedBuildInputs ? []`: Aside from propagating dependencies,
1222- `buildPythonPackage` also injects code into and wraps executables with the
1223- paths included in this list. Items listed in `install_requires` go here.
1224-1225-##### Overriding Python packages {#overriding-python-packages}
1226-1227-The `buildPythonPackage` function has a `overridePythonAttrs` method that can be
1228-used to override the package. In the following example we create an environment
1229-where we have the `blaze` package using an older version of `pandas`. We
1230-override first the Python interpreter and pass `packageOverrides` which contains
1231-the overrides for packages in the package set.
1232-1233-```nix
1234-with import <nixpkgs> {};
1235-1236-(let
1237- python = let
1238- packageOverrides = self: super: {
1239- pandas = super.pandas.overridePythonAttrs(old: rec {
1240- version = "0.19.1";
1241- src = fetchPypi {
1242- pname = "pandas";
1243- inherit version;
1244- hash = "sha256-JQn+rtpy/OA2deLszSKEuxyttqBzcAil50H+JDHUdCE=";
1245- };
1246- });
1247- };
1248- in pkgs.python3.override {inherit packageOverrides; self = python;};
1249-1250-in python.withPackages(ps: [ ps.blaze ])).env
1251-```
1252-1253-The next example shows a non trivial overriding of the `blas` implementation to
1254-be used through out all of the Python package set:
1255-1256-```nix
1257-python3MyBlas = pkgs.python3.override {
1258- packageOverrides = self: super: {
1259- # We need toPythonModule for the package set to evaluate this
1260- blas = super.toPythonModule(super.pkgs.blas.override {
1261- blasProvider = super.pkgs.mkl;
1262- });
1263- lapack = super.toPythonModule(super.pkgs.lapack.override {
1264- lapackProvider = super.pkgs.mkl;
1265- });
1266- };
1267-};
1268-```
1269-1270-This is particularly useful for numpy and scipy users who want to gain speed with other blas implementations.
1271-Note that using simply `scipy = super.scipy.override { blas = super.pkgs.mkl; };` will likely result in
1272-compilation issues, because scipy dependencies need to use the same blas implementation as well.
1273-1274-#### Optional extra dependencies {#python-optional-dependencies}
1275-1276-Some packages define optional dependencies for additional features. With
1277-`setuptools` this is called `extras_require` and `flit` calls it
1278-`extras-require`, while PEP 621 calls these `optional-dependencies`. A
1279-method for supporting this is by declaring the extras of a package in its
1280-`passthru`, e.g. in case of the package `dask`
1281-1282-```nix
1283-passthru.optional-dependencies = {
1284- complete = [ distributed ];
1285-};
1286-```
1287-1288-and letting the package requiring the extra add the list to its dependencies
1289-1290-```nix
1291-propagatedBuildInputs = [
1292- ...
1293-] ++ dask.optional-dependencies.complete;
1294-```
1295-1296-Note this method is preferred over adding parameters to builders, as that can
1297-result in packages depending on different variants and thereby causing
1298-collisions.
1299-1300-#### `buildPythonApplication` function {#buildpythonapplication-function}
1301-1302-The `buildPythonApplication` function is practically the same as
1303-`buildPythonPackage`. The main purpose of this function is to build a Python
1304-package where one is interested only in the executables, and not importable
1305-modules. For that reason, when adding this package to a `python.buildEnv`, the
1306-modules won't be made available.
1307-1308-Another difference is that `buildPythonPackage` by default prefixes the names of
1309-the packages with the version of the interpreter. Because this is irrelevant for
1310-applications, the prefix is omitted.
1311-1312-When packaging a Python application with `buildPythonApplication`, it should be
1313-called with `callPackage` and passed `python` or `pythonPackages` (possibly
1314-specifying an interpreter version), like this:
1315-1316-```nix
1317-{ lib
1318-, python3
1319-, fetchPypi
1320-}:
1321-1322-python3.pkgs.buildPythonApplication rec {
1323- pname = "luigi";
1324- version = "2.7.9";
1325- format = "setuptools";
1326-1327- src = fetchPypi {
1328- inherit pname version;
1329- hash = "sha256-Pe229rT0aHwA98s+nTHQMEFKZPo/yw6sot8MivFDvAw=";
1330- };
1331-1332- propagatedBuildInputs = with python3.pkgs; [
1333- tornado
1334- python-daemon
1335- ];
1336-1337- meta = with lib; {
1338- ...
1339- };
1340-}
1341-```
1342-1343-This is then added to `all-packages.nix` just as any other application would be.
1344-1345-```nix
1346-luigi = callPackage ../applications/networking/cluster/luigi { };
1347-```
1348-1349-Since the package is an application, a consumer doesn't need to care about
1350-Python versions or modules, which is why they don't go in `pythonPackages`.
1351-1352-#### `toPythonApplication` function {#topythonapplication-function}
1353-1354-A distinction is made between applications and libraries, however, sometimes a
1355-package is used as both. In this case the package is added as a library to
1356-`python-packages.nix` and as an application to `all-packages.nix`. To reduce
1357-duplication the `toPythonApplication` can be used to convert a library to an
1358-application.
1359-1360-The Nix expression shall use `buildPythonPackage` and be called from
1361-`python-packages.nix`. A reference shall be created from `all-packages.nix` to
1362-the attribute in `python-packages.nix`, and the `toPythonApplication` shall be
1363-applied to the reference:
1364-1365-```nix
1366-youtube-dl = with pythonPackages; toPythonApplication youtube-dl;
1367-```
1368-1369-#### `toPythonModule` function {#topythonmodule-function}
1370-1371-In some cases, such as bindings, a package is created using
1372-`stdenv.mkDerivation` and added as attribute in `all-packages.nix`. The Python
1373-bindings should be made available from `python-packages.nix`. The
1374-`toPythonModule` function takes a derivation and makes certain Python-specific
1375-modifications.
1376-1377-```nix
1378-opencv = toPythonModule (pkgs.opencv.override {
1379- enablePython = true;
1380- pythonPackages = self;
1381-});
1382-```
1383-1384-Do pay attention to passing in the right Python version!
1385-1386-#### `python.buildEnv` function {#python.buildenv-function}
1387-1388-Python environments can be created using the low-level `pkgs.buildEnv` function.
1389-This example shows how to create an environment that has the Pyramid Web Framework.
1390-Saving the following as `default.nix`
1391-1392-```nix
1393-with import <nixpkgs> {};
1394-1395-python.buildEnv.override {
1396- extraLibs = [ pythonPackages.pyramid ];
1397- ignoreCollisions = true;
1398-}
1399-```
1400-1401-and running `nix-build` will create
1402-1403-```
1404-/nix/store/cf1xhjwzmdki7fasgr4kz6di72ykicl5-python-2.7.8-env
1405-```
1406-1407-with wrapped binaries in `bin/`.
1408-1409-You can also use the `env` attribute to create local environments with needed
1410-packages installed. This is somewhat comparable to `virtualenv`. For example,
1411-running `nix-shell` with the following `shell.nix`
1412-1413-```nix
1414-with import <nixpkgs> {};
1415-1416-(python3.buildEnv.override {
1417- extraLibs = with python3Packages; [
1418- numpy
1419- requests
1420- ];
1421-}).env
1422-```
1423-1424-will drop you into a shell where Python will have the
1425-specified packages in its path.
1426-1427-##### `python.buildEnv` arguments {#python.buildenv-arguments}
1428-1429-1430-* `extraLibs`: List of packages installed inside the environment.
1431-* `postBuild`: Shell command executed after the build of environment.
1432-* `ignoreCollisions`: Ignore file collisions inside the environment (default is `false`).
1433-* `permitUserSite`: Skip setting the `PYTHONNOUSERSITE` environment variable in
1434- wrapped binaries in the environment.
1435-1436-#### `python.withPackages` function {#python.withpackages-function}
1437-1438-The `python.withPackages` function provides a simpler interface to the `python.buildEnv` functionality.
1439-It takes a function as an argument that is passed the set of python packages and returns the list
1440-of the packages to be included in the environment. Using the `withPackages` function, the previous
1441-example for the Pyramid Web Framework environment can be written like this:
1442-1443-```nix
1444-with import <nixpkgs> {};
1445-1446-python.withPackages (ps: [ ps.pyramid ])
1447-```
1448-1449-`withPackages` passes the correct package set for the specific interpreter
1450-version as an argument to the function. In the above example, `ps` equals
1451-`pythonPackages`. But you can also easily switch to using python3:
1452-1453-```nix
1454-with import <nixpkgs> {};
1455-1456-python3.withPackages (ps: [ ps.pyramid ])
1457-```
1458-1459-Now, `ps` is set to `python3Packages`, matching the version of the interpreter.
1460-1461-As `python.withPackages` simply uses `python.buildEnv` under the hood, it also
1462-supports the `env` attribute. The `shell.nix` file from the previous section can
1463-thus be also written like this:
1464-1465-```nix
1466-with import <nixpkgs> {};
1467-1468-(python3.withPackages (ps: with ps; [
1469- numpy
1470- requests
1471-])).env
1472-```
1473-1474-In contrast to `python.buildEnv`, `python.withPackages` does not support the
1475-more advanced options such as `ignoreCollisions = true` or `postBuild`. If you
1476-need them, you have to use `python.buildEnv`.
1477-1478-Python 2 namespace packages may provide `__init__.py` that collide. In that case
1479-`python.buildEnv` should be used with `ignoreCollisions = true`.
1480-1481-#### Setup hooks {#setup-hooks}
1482-1483-The following are setup hooks specifically for Python packages. Most of these
1484-are used in `buildPythonPackage`.
1485-1486-- `eggUnpackhook` to move an egg to the correct folder so it can be installed
1487- with the `eggInstallHook`
1488-- `eggBuildHook` to skip building for eggs.
1489-- `eggInstallHook` to install eggs.
1490-- `flitBuildHook` to build a wheel using `flit`.
1491-- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system
1492- (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
1493-- `pypaBuildHook` to build a wheel using
1494- [`pypa/build`](https://pypa-build.readthedocs.io/en/latest/index.html) and
1495- PEP 517/518. Note a build system (e.g. `setuptools` or `flit`) should still
1496- be added as `nativeBuildInput`.
1497-- `pipInstallHook` to install wheels.
1498-- `pytestCheckHook` to run tests with `pytest`. See [example usage](#using-pytestcheckhook).
1499-- `pythonCatchConflictsHook` to check whether a Python package is not already existing.
1500-- `pythonImportsCheckHook` to check whether importing the listed modules works.
1501-- `pythonRelaxDepsHook` will relax Python dependencies restrictions for the package.
1502- See [example usage](#using-pythonrelaxdepshook).
1503-- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
1504-- `setuptoolsBuildHook` to build a wheel using `setuptools`.
1505-- `setuptoolsCheckHook` to run tests with `python setup.py test`.
1506-- `sphinxHook` to build documentation and manpages using Sphinx.
1507-- `venvShellHook` to source a Python 3 `venv` at the `venvDir` location. A
1508- `venv` is created if it does not yet exist. `postVenvCreation` can be used to
1509- to run commands only after venv is first created.
1510-- `wheelUnpackHook` to move a wheel to the correct folder so it can be installed
1511- with the `pipInstallHook`.
1512-- `unittestCheckHook` will run tests with `python -m unittest discover`. See [example usage](#using-unittestcheckhook).
1513-1514-### Development mode {#development-mode}
1515-1516-Development or editable mode is supported. To develop Python packages
1517-`buildPythonPackage` has additional logic inside `shellPhase` to run `pip
1518-install -e . --prefix $TMPDIR/`for the package.
1519-1520-Warning: `shellPhase` is executed only if `setup.py` exists.
1521-1522-Given a `default.nix`:
1523-1524-```nix
1525-with import <nixpkgs> {};
1526-1527-pythonPackages.buildPythonPackage {
1528- name = "myproject";
1529- buildInputs = with pythonPackages; [ pyramid ];
1530-1531- src = ./.;
1532-}
1533-```
1534-1535-Running `nix-shell` with no arguments should give you the environment in which
1536-the package would be built with `nix-build`.
1537-1538-Shortcut to setup environments with C headers/libraries and Python packages:
1539-1540-```shell
1541-nix-shell -p pythonPackages.pyramid zlib libjpeg git
1542-```
1543-1544-Note: There is a boolean value `lib.inNixShell` set to `true` if nix-shell is invoked.
1545-1546-### Tools {#tools}
1547-1548-Packages inside nixpkgs must use the `buildPythonPackage` or `buildPythonApplication` function directly,
1549-because we can only provide security support for non-vendored dependencies.
1550-1551-We recommend [nix-init](https://github.com/nix-community/nix-init) for creating new python packages within nixpkgs,
1552-as it already prefetches the source, parses dependencies for common formats and prefills most things in `meta`.
1553-1554-### Deterministic builds {#deterministic-builds}
1555-1556-The Python interpreters are now built deterministically. Minor modifications had
1557-to be made to the interpreters in order to generate deterministic bytecode. This
1558-has security implications and is relevant for those using Python in a
1559-`nix-shell`.
1560-1561-When the environment variable `DETERMINISTIC_BUILD` is set, all bytecode will
1562-have timestamp 1. The `buildPythonPackage` function sets `DETERMINISTIC_BUILD=1`
1563-and [PYTHONHASHSEED=0](https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONHASHSEED).
1564-Both are also exported in `nix-shell`.
1565-1566-### Automatic tests {#automatic-tests}
1567-1568-It is recommended to test packages as part of the build process.
1569-Source distributions (`sdist`) often include test files, but not always.
1570-1571-By default the command `python setup.py test` is run as part of the
1572-`checkPhase`, but often it is necessary to pass a custom `checkPhase`. An
1573-example of such a situation is when `py.test` is used.
1574-1575-#### Common issues {#common-issues}
1576-1577-* Non-working tests can often be deselected. By default `buildPythonPackage`
1578- runs `python setup.py test`. which is deprecated. Most Python modules however
1579- do follow the standard test protocol where the pytest runner can be used
1580- instead. `pytest` supports the `-k` and `--ignore` parameters to ignore test
1581- methods or classes as well as whole files. For `pytestCheckHook` these are
1582- conveniently exposed as `disabledTests` and `disabledTestPaths` respectively.
1583-1584- ```nix
1585- buildPythonPackage {
1586- # ...
1587- nativeCheckInputs = [
1588- pytestCheckHook
1589- ];
1590-1591- disabledTests = [
1592- "function_name"
1593- "other_function"
1594- ];
1595-1596- disabledTestPaths = [
1597- "this/file.py"
1598- ];
1599- }
1600- ```
1601-1602-* Tests that attempt to access `$HOME` can be fixed by using the following
1603- work-around before running tests (e.g. `preCheck`): `export HOME=$(mktemp -d)`
1604-1605## FAQ {#faq}
16061607### How to solve circular dependencies? {#how-to-solve-circular-dependencies}
···1949* `setup_requires` corresponds to `nativeBuildInputs`
1950* `install_requires` corresponds to `propagatedBuildInputs`
1951* `tests_require` corresponds to `nativeCheckInputs`
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000019521953## Contributing {#contributing}
1954
···1# Python {#python}
23+## Reference {#reference}
4+5+### Interpreters {#interpreters}
6+7+| Package | Aliases | Interpreter |
8+|------------|-----------------|-------------|
9+| python27 | python2, python | CPython 2.7 |
10+| python38 | | CPython 3.8 |
11+| python39 | | CPython 3.9 |
12+| python310 | python3 | CPython 3.10 |
13+| python311 | | CPython 3.11 |
14+| python312 | | CPython 3.12 |
15+| pypy27 | pypy2, pypy | PyPy2.7 |
16+| pypy39 | pypy3 | PyPy 3.9 |
17+18+The Nix expressions for the interpreters can be found in
19+`pkgs/development/interpreters/python`.
20+21+All packages depending on any Python interpreter get appended
22+`out/{python.sitePackages}` to `$PYTHONPATH` if such directory
23+exists.
24+25+#### Missing `tkinter` module standard library {#missing-tkinter-module-standard-library}
26+27+To reduce closure size the `Tkinter`/`tkinter` is available as a separate package, `pythonPackages.tkinter`.
28+29+#### Attributes on interpreters packages {#attributes-on-interpreters-packages}
30+31+Each interpreter has the following attributes:
32+33+- `libPrefix`. Name of the folder in `${python}/lib/` for corresponding interpreter.
34+- `interpreter`. Alias for `${python}/bin/${executable}`.
35+- `buildEnv`. Function to build python interpreter environments with extra packages bundled together. See section *python.buildEnv function* for usage and documentation.
36+- `withPackages`. Simpler interface to `buildEnv`. See section *python.withPackages function* for usage and documentation.
37+- `sitePackages`. Alias for `lib/${libPrefix}/site-packages`.
38+- `executable`. Name of the interpreter executable, e.g. `python3.10`.
39+- `pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
40+41+### Building packages and applications {#building-packages-and-applications}
42+43+Python libraries and applications that use `setuptools` or
44+`distutils` are typically built with respectively the `buildPythonPackage` and
45+`buildPythonApplication` functions. These two functions also support installing a `wheel`.
46+47+All Python packages reside in `pkgs/top-level/python-packages.nix` and all
48+applications elsewhere. In case a package is used as both a library and an
49+application, then the package should be in `pkgs/top-level/python-packages.nix`
50+since only those packages are made available for all interpreter versions. The
51+preferred location for library expressions is in
52+`pkgs/development/python-modules`. It is important that these packages are
53+called from `pkgs/top-level/python-packages.nix` and not elsewhere, to guarantee
54+the right version of the package is built.
55+56+Based on the packages defined in `pkgs/top-level/python-packages.nix` an
57+attribute set is created for each available Python interpreter. The available
58+sets are
59+60+* `pkgs.python27Packages`
61+* `pkgs.python3Packages`
62+* `pkgs.python38Packages`
63+* `pkgs.python39Packages`
64+* `pkgs.python310Packages`
65+* `pkgs.python311Packages`
66+* `pkgs.pypyPackages`
67+68+and the aliases
69+70+* `pkgs.python2Packages` pointing to `pkgs.python27Packages`
71+* `pkgs.python3Packages` pointing to `pkgs.python310Packages`
72+* `pkgs.pythonPackages` pointing to `pkgs.python2Packages`
73+74+#### `buildPythonPackage` function {#buildpythonpackage-function}
75+76+The `buildPythonPackage` function is implemented in
77+`pkgs/development/interpreters/python/mk-python-derivation.nix`
78+using setup hooks.
79+80+The following is an example:
81+82+```nix
83+{ lib
84+, buildPythonPackage
85+, fetchPypi
86+87+# build-system
88+, setuptools-scm
89+90+# dependencies
91+, attrs
92+, pluggy
93+, py
94+, setuptools
95+, six
96+97+# tests
98+, hypothesis
99+ }:
100+101+buildPythonPackage rec {
102+ pname = "pytest";
103+ version = "3.3.1";
104+ format = "setuptools";
105+106+ src = fetchPypi {
107+ inherit pname version;
108+ hash = "sha256-z4Q23FnYaVNG/NOrKW3kZCXsqwDWQJbOvnn7Ueyy65M=";
109+ };
110+111+ postPatch = ''
112+ # don't test bash builtins
113+ rm testing/test_argcomplete.py
114+ '';
115+116+ nativeBuildInputs = [
117+ setuptools-scm
118+ ];
119+120+ propagatedBuildInputs = [
121+ attrs
122+ py
123+ setuptools
124+ six
125+ pluggy
126+ ];
127+128+ nativeCheckInputs = [
129+ hypothesis
130+ ];
131+132+ meta = with lib; {
133+ changelog = "https://github.com/pytest-dev/pytest/releases/tag/${version}";
134+ description = "Framework for writing tests";
135+ homepage = "https://github.com/pytest-dev/pytest";
136+ license = licenses.mit;
137+ maintainers = with maintainers; [ domenkozar lovek323 madjar lsix ];
138+ };
139+}
140+```
141+142+The `buildPythonPackage` mainly does four things:
143+144+* In the `buildPhase`, it calls `${python.pythonForBuild.interpreter} setup.py bdist_wheel` to
145+ build a wheel binary zipfile.
146+* In the `installPhase`, it installs the wheel file using `pip install *.whl`.
147+* In the `postFixup` phase, the `wrapPythonPrograms` bash function is called to
148+ wrap all programs in the `$out/bin/*` directory to include `$PATH`
149+ environment variable and add dependent libraries to script's `sys.path`.
150+* In the `installCheck` phase, `${python.interpreter} setup.py test` is run.
151+152+By default tests are run because `doCheck = true`. Test dependencies, like
153+e.g. the test runner, should be added to `nativeCheckInputs`.
154+155+By default `meta.platforms` is set to the same value
156+as the interpreter unless overridden otherwise.
157+158+##### `buildPythonPackage` parameters {#buildpythonpackage-parameters}
159+160+All parameters from `stdenv.mkDerivation` function are still supported. The
161+following are specific to `buildPythonPackage`:
162+163+* `catchConflicts ? true`: If `true`, abort package build if a package name
164+ appears more than once in dependency tree. Default is `true`.
165+* `disabled ? false`: If `true`, package is not built for the particular Python
166+ interpreter version.
167+* `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
168+* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
169+ variable in wrapped programs.
170+* `format ? "setuptools"`: Format of the source. Valid options are
171+ `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`.
172+ `"setuptools"` is for when the source has a `setup.py` and `setuptools` is
173+ used to build a wheel, `flit`, in case `flit` should be used to build a wheel,
174+ and `wheel` in case a wheel is provided. Use `other` when a custom
175+ `buildPhase` and/or `installPhase` is needed.
176+* `makeWrapperArgs ? []`: A list of strings. Arguments to be passed to
177+ `makeWrapper`, which wraps generated binaries. By default, the arguments to
178+ `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling
179+ the binary. Additional arguments here can allow a developer to set environment
180+ variables which will be available when the binary is run. For example,
181+ `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
182+* `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this
183+ defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications to `""`.
184+* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip
185+ install`. To pass options to `python setup.py install`, use
186+ `--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`.
187+* `pipBuildFlags ? []`: A list of strings. Arguments to be passed to `pip wheel`.
188+* `pypaBuildFlags ? []`: A list of strings. Arguments to be passed to `python -m build --wheel`.
189+* `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages
190+ in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`).
191+* `preShellHook`: Hook to execute commands before `shellHook`.
192+* `postShellHook`: Hook to execute commands after `shellHook`.
193+* `removeBinByteCode ? true`: Remove bytecode from `/bin`. Bytecode is only
194+ created when the filenames end with `.py`.
195+* `setupPyGlobalFlags ? []`: List of flags passed to `setup.py` command.
196+* `setupPyBuildFlags ? []`: List of flags passed to `setup.py build_ext` command.
197+198+The `stdenv.mkDerivation` function accepts various parameters for describing
199+build inputs (see "Specifying dependencies"). The following are of special
200+interest for Python packages, either because these are primarily used, or
201+because their behaviour is different:
202+203+* `nativeBuildInputs ? []`: Build-time only dependencies. Typically executables
204+ as well as the items listed in `setup_requires`.
205+* `buildInputs ? []`: Build and/or run-time dependencies that need to be
206+ compiled for the host machine. Typically non-Python libraries which are being
207+ linked.
208+* `nativeCheckInputs ? []`: Dependencies needed for running the `checkPhase`. These
209+ are added to `nativeBuildInputs` when `doCheck = true`. Items listed in
210+ `tests_require` go here.
211+* `propagatedBuildInputs ? []`: Aside from propagating dependencies,
212+ `buildPythonPackage` also injects code into and wraps executables with the
213+ paths included in this list. Items listed in `install_requires` go here.
214+215+##### Overriding Python packages {#overriding-python-packages}
216+217+The `buildPythonPackage` function has a `overridePythonAttrs` method that can be
218+used to override the package. In the following example we create an environment
219+where we have the `blaze` package using an older version of `pandas`. We
220+override first the Python interpreter and pass `packageOverrides` which contains
221+the overrides for packages in the package set.
222+223+```nix
224+with import <nixpkgs> {};
225+226+(let
227+ python = let
228+ packageOverrides = self: super: {
229+ pandas = super.pandas.overridePythonAttrs(old: rec {
230+ version = "0.19.1";
231+ src = fetchPypi {
232+ pname = "pandas";
233+ inherit version;
234+ hash = "sha256-JQn+rtpy/OA2deLszSKEuxyttqBzcAil50H+JDHUdCE=";
235+ };
236+ });
237+ };
238+ in pkgs.python3.override {inherit packageOverrides; self = python;};
239+240+in python.withPackages(ps: [ ps.blaze ])).env
241+```
242+243+The next example shows a non trivial overriding of the `blas` implementation to
244+be used through out all of the Python package set:
245+246+```nix
247+python3MyBlas = pkgs.python3.override {
248+ packageOverrides = self: super: {
249+ # We need toPythonModule for the package set to evaluate this
250+ blas = super.toPythonModule(super.pkgs.blas.override {
251+ blasProvider = super.pkgs.mkl;
252+ });
253+ lapack = super.toPythonModule(super.pkgs.lapack.override {
254+ lapackProvider = super.pkgs.mkl;
255+ });
256+ };
257+};
258+```
259+260+This is particularly useful for numpy and scipy users who want to gain speed with other blas implementations.
261+Note that using simply `scipy = super.scipy.override { blas = super.pkgs.mkl; };` will likely result in
262+compilation issues, because scipy dependencies need to use the same blas implementation as well.
263+264+#### `buildPythonApplication` function {#buildpythonapplication-function}
265+266+The `buildPythonApplication` function is practically the same as
267+`buildPythonPackage`. The main purpose of this function is to build a Python
268+package where one is interested only in the executables, and not importable
269+modules. For that reason, when adding this package to a `python.buildEnv`, the
270+modules won't be made available.
271+272+Another difference is that `buildPythonPackage` by default prefixes the names of
273+the packages with the version of the interpreter. Because this is irrelevant for
274+applications, the prefix is omitted.
275+276+When packaging a Python application with `buildPythonApplication`, it should be
277+called with `callPackage` and passed `python` or `pythonPackages` (possibly
278+specifying an interpreter version), like this:
279+280+```nix
281+{ lib
282+, python3
283+, fetchPypi
284+}:
285+286+python3.pkgs.buildPythonApplication rec {
287+ pname = "luigi";
288+ version = "2.7.9";
289+ format = "setuptools";
290+291+ src = fetchPypi {
292+ inherit pname version;
293+ hash = "sha256-Pe229rT0aHwA98s+nTHQMEFKZPo/yw6sot8MivFDvAw=";
294+ };
295+296+ propagatedBuildInputs = with python3.pkgs; [
297+ tornado
298+ python-daemon
299+ ];
300+301+ meta = with lib; {
302+ ...
303+ };
304+}
305+```
306+307+This is then added to `all-packages.nix` just as any other application would be.
308+309+```nix
310+luigi = callPackage ../applications/networking/cluster/luigi { };
311+```
312+313+Since the package is an application, a consumer doesn't need to care about
314+Python versions or modules, which is why they don't go in `pythonPackages`.
315+316+#### `toPythonApplication` function {#topythonapplication-function}
317+318+A distinction is made between applications and libraries, however, sometimes a
319+package is used as both. In this case the package is added as a library to
320+`python-packages.nix` and as an application to `all-packages.nix`. To reduce
321+duplication the `toPythonApplication` can be used to convert a library to an
322+application.
323+324+The Nix expression shall use `buildPythonPackage` and be called from
325+`python-packages.nix`. A reference shall be created from `all-packages.nix` to
326+the attribute in `python-packages.nix`, and the `toPythonApplication` shall be
327+applied to the reference:
328+329+```nix
330+youtube-dl = with pythonPackages; toPythonApplication youtube-dl;
331+```
332+333+#### `toPythonModule` function {#topythonmodule-function}
334+335+In some cases, such as bindings, a package is created using
336+`stdenv.mkDerivation` and added as attribute in `all-packages.nix`. The Python
337+bindings should be made available from `python-packages.nix`. The
338+`toPythonModule` function takes a derivation and makes certain Python-specific
339+modifications.
340+341+```nix
342+opencv = toPythonModule (pkgs.opencv.override {
343+ enablePython = true;
344+ pythonPackages = self;
345+});
346+```
347+348+Do pay attention to passing in the right Python version!
349+350+#### `python.buildEnv` function {#python.buildenv-function}
351+352+Python environments can be created using the low-level `pkgs.buildEnv` function.
353+This example shows how to create an environment that has the Pyramid Web Framework.
354+Saving the following as `default.nix`
355+356+```nix
357+with import <nixpkgs> {};
358+359+python.buildEnv.override {
360+ extraLibs = [ pythonPackages.pyramid ];
361+ ignoreCollisions = true;
362+}
363+```
364+365+and running `nix-build` will create
366+367+```
368+/nix/store/cf1xhjwzmdki7fasgr4kz6di72ykicl5-python-2.7.8-env
369+```
370+371+with wrapped binaries in `bin/`.
372+373+You can also use the `env` attribute to create local environments with needed
374+packages installed. This is somewhat comparable to `virtualenv`. For example,
375+running `nix-shell` with the following `shell.nix`
376+377+```nix
378+with import <nixpkgs> {};
379+380+(python3.buildEnv.override {
381+ extraLibs = with python3Packages; [
382+ numpy
383+ requests
384+ ];
385+}).env
386+```
387+388+will drop you into a shell where Python will have the
389+specified packages in its path.
390+391+##### `python.buildEnv` arguments {#python.buildenv-arguments}
392+393+394+* `extraLibs`: List of packages installed inside the environment.
395+* `postBuild`: Shell command executed after the build of environment.
396+* `ignoreCollisions`: Ignore file collisions inside the environment (default is `false`).
397+* `permitUserSite`: Skip setting the `PYTHONNOUSERSITE` environment variable in
398+ wrapped binaries in the environment.
399+400+#### `python.withPackages` function {#python.withpackages-function}
401+402+The `python.withPackages` function provides a simpler interface to the `python.buildEnv` functionality.
403+It takes a function as an argument that is passed the set of python packages and returns the list
404+of the packages to be included in the environment. Using the `withPackages` function, the previous
405+example for the Pyramid Web Framework environment can be written like this:
406+407+```nix
408+with import <nixpkgs> {};
409+410+python.withPackages (ps: [ ps.pyramid ])
411+```
412+413+`withPackages` passes the correct package set for the specific interpreter
414+version as an argument to the function. In the above example, `ps` equals
415+`pythonPackages`. But you can also easily switch to using python3:
416+417+```nix
418+with import <nixpkgs> {};
419+420+python3.withPackages (ps: [ ps.pyramid ])
421+```
422+423+Now, `ps` is set to `python3Packages`, matching the version of the interpreter.
424+425+As `python.withPackages` simply uses `python.buildEnv` under the hood, it also
426+supports the `env` attribute. The `shell.nix` file from the previous section can
427+thus be also written like this:
428+429+```nix
430+with import <nixpkgs> {};
431+432+(python3.withPackages (ps: with ps; [
433+ numpy
434+ requests
435+])).env
436+```
437+438+In contrast to `python.buildEnv`, `python.withPackages` does not support the
439+more advanced options such as `ignoreCollisions = true` or `postBuild`. If you
440+need them, you have to use `python.buildEnv`.
441+442+Python 2 namespace packages may provide `__init__.py` that collide. In that case
443+`python.buildEnv` should be used with `ignoreCollisions = true`.
444+445+#### Setup hooks {#setup-hooks}
446+447+The following are setup hooks specifically for Python packages. Most of these
448+are used in `buildPythonPackage`.
449+450+- `eggUnpackhook` to move an egg to the correct folder so it can be installed
451+ with the `eggInstallHook`
452+- `eggBuildHook` to skip building for eggs.
453+- `eggInstallHook` to install eggs.
454+- `flitBuildHook` to build a wheel using `flit`.
455+- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system
456+ (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
457+- `pypaBuildHook` to build a wheel using
458+ [`pypa/build`](https://pypa-build.readthedocs.io/en/latest/index.html) and
459+ PEP 517/518. Note a build system (e.g. `setuptools` or `flit`) should still
460+ be added as `nativeBuildInput`.
461+- `pipInstallHook` to install wheels.
462+- `pytestCheckHook` to run tests with `pytest`. See [example usage](#using-pytestcheckhook).
463+- `pythonCatchConflictsHook` to check whether a Python package is not already existing.
464+- `pythonImportsCheckHook` to check whether importing the listed modules works.
465+- `pythonRelaxDepsHook` will relax Python dependencies restrictions for the package.
466+ See [example usage](#using-pythonrelaxdepshook).
467+- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
468+- `setuptoolsBuildHook` to build a wheel using `setuptools`.
469+- `setuptoolsCheckHook` to run tests with `python setup.py test`.
470+- `sphinxHook` to build documentation and manpages using Sphinx.
471+- `venvShellHook` to source a Python 3 `venv` at the `venvDir` location. A
472+ `venv` is created if it does not yet exist. `postVenvCreation` can be used to
473+ to run commands only after venv is first created.
474+- `wheelUnpackHook` to move a wheel to the correct folder so it can be installed
475+ with the `pipInstallHook`.
476+- `unittestCheckHook` will run tests with `python -m unittest discover`. See [example usage](#using-unittestcheckhook).
477+478+### Development mode {#development-mode}
479+480+Development or editable mode is supported. To develop Python packages
481+`buildPythonPackage` has additional logic inside `shellPhase` to run `pip
482+install -e . --prefix $TMPDIR/`for the package.
483+484+Warning: `shellPhase` is executed only if `setup.py` exists.
485+486+Given a `default.nix`:
487+488+```nix
489+with import <nixpkgs> {};
490+491+pythonPackages.buildPythonPackage {
492+ name = "myproject";
493+ buildInputs = with pythonPackages; [ pyramid ];
494+495+ src = ./.;
496+}
497+```
498+499+Running `nix-shell` with no arguments should give you the environment in which
500+the package would be built with `nix-build`.
501+502+Shortcut to setup environments with C headers/libraries and Python packages:
503+504+```shell
505+nix-shell -p pythonPackages.pyramid zlib libjpeg git
506+```
507+508+Note: There is a boolean value `lib.inNixShell` set to `true` if nix-shell is invoked.
509+510## User Guide {#user-guide}
511512### Using Python {#using-python}
···1500example we use `buildPythonPackage` that is part of the set `python3Packages`,
1501and in this case the `python3` interpreter is automatically used.
15020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001503## FAQ {#faq}
15041505### How to solve circular dependencies? {#how-to-solve-circular-dependencies}
···1847* `setup_requires` corresponds to `nativeBuildInputs`
1848* `install_requires` corresponds to `propagatedBuildInputs`
1849* `tests_require` corresponds to `nativeCheckInputs`
1850+1851+### How to enable interpreter optimizations? {#optimizations}
1852+1853+The Python interpreters are by default not built with optimizations enabled, because
1854+the builds are in that case not reproducible. To enable optimizations, override the
1855+interpreter of interest, e.g using
1856+1857+```
1858+let
1859+ pkgs = import ./. {};
1860+ mypython = pkgs.python3.override {
1861+ enableOptimizations = true;
1862+ reproducibleBuild = false;
1863+ self = mypython;
1864+ };
1865+in mypython
1866+```
1867+1868+### How to add optional dependencies? {#python-optional-dependencies}
1869+1870+Some packages define optional dependencies for additional features. With
1871+`setuptools` this is called `extras_require` and `flit` calls it
1872+`extras-require`, while PEP 621 calls these `optional-dependencies`. A
1873+method for supporting this is by declaring the extras of a package in its
1874+`passthru`, e.g. in case of the package `dask`
1875+1876+```nix
1877+passthru.optional-dependencies = {
1878+ complete = [ distributed ];
1879+};
1880+```
1881+1882+and letting the package requiring the extra add the list to its dependencies
1883+1884+```nix
1885+propagatedBuildInputs = [
1886+ ...
1887+] ++ dask.optional-dependencies.complete;
1888+```
1889+1890+Note this method is preferred over adding parameters to builders, as that can
1891+result in packages depending on different variants and thereby causing
1892+collisions.
1893+1894+### How to contribute a Python package to nixpkgs? {#tools}
1895+1896+Packages inside nixpkgs must use the `buildPythonPackage` or `buildPythonApplication` function directly,
1897+because we can only provide security support for non-vendored dependencies.
1898+1899+We recommend [nix-init](https://github.com/nix-community/nix-init) for creating new python packages within nixpkgs,
1900+as it already prefetches the source, parses dependencies for common formats and prefills most things in `meta`.
1901+1902+### Are Python interpreters built deterministically? {#deterministic-builds}
1903+1904+The Python interpreters are now built deterministically. Minor modifications had
1905+to be made to the interpreters in order to generate deterministic bytecode. This
1906+has security implications and is relevant for those using Python in a
1907+`nix-shell`.
1908+1909+When the environment variable `DETERMINISTIC_BUILD` is set, all bytecode will
1910+have timestamp 1. The `buildPythonPackage` function sets `DETERMINISTIC_BUILD=1`
1911+and [PYTHONHASHSEED=0](https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONHASHSEED).
1912+Both are also exported in `nix-shell`.
1913+1914+### How to provide automatic tests to Python packages? {#automatic-tests}
1915+1916+It is recommended to test packages as part of the build process.
1917+Source distributions (`sdist`) often include test files, but not always.
1918+1919+By default the command `python setup.py test` is run as part of the
1920+`checkPhase`, but often it is necessary to pass a custom `checkPhase`. An
1921+example of such a situation is when `py.test` is used.
1922+1923+#### Common issues {#common-issues}
1924+1925+* Non-working tests can often be deselected. By default `buildPythonPackage`
1926+ runs `python setup.py test`. which is deprecated. Most Python modules however
1927+ do follow the standard test protocol where the pytest runner can be used
1928+ instead. `pytest` supports the `-k` and `--ignore` parameters to ignore test
1929+ methods or classes as well as whole files. For `pytestCheckHook` these are
1930+ conveniently exposed as `disabledTests` and `disabledTestPaths` respectively.
1931+1932+ ```nix
1933+ buildPythonPackage {
1934+ # ...
1935+ nativeCheckInputs = [
1936+ pytestCheckHook
1937+ ];
1938+1939+ disabledTests = [
1940+ "function_name"
1941+ "other_function"
1942+ ];
1943+1944+ disabledTestPaths = [
1945+ "this/file.py"
1946+ ];
1947+ }
1948+ ```
1949+1950+* Tests that attempt to access `$HOME` can be fixed by using the following
1951+ work-around before running tests (e.g. `preCheck`): `export HOME=$(mktemp -d)`
19521953## Contributing {#contributing}
1954
···361362 # Optional features:
363 use_gio = true;
364- use_gnome_keyring = false; # Superseded by libsecret
365 use_cups = cupsSupport;
366367 # Feature overrides:
···384 # We do intentionally not set rustc_version as nixpkgs will never do incremental
385 # rebuilds, thus leaving this empty is fine.
386 rust_sysroot_absolute = "${rustc}";
00387 } // lib.optionalAttrs (!(stdenv.buildPlatform.canExecute stdenv.hostPlatform)) {
388 # https://www.mail-archive.com/v8-users@googlegroups.com/msg14528.html
389 arm_control_flow_integrity = "none";
···361362 # Optional features:
363 use_gio = true;
0364 use_cups = cupsSupport;
365366 # Feature overrides:
···383 # We do intentionally not set rustc_version as nixpkgs will never do incremental
384 # rebuilds, thus leaving this empty is fine.
385 rust_sysroot_absolute = "${rustc}";
386+ # Building with rust is disabled for now - this matches the flags in other major distributions.
387+ enable_rust = false;
388 } // lib.optionalAttrs (!(stdenv.buildPlatform.canExecute stdenv.hostPlatform)) {
389 # https://www.mail-archive.com/v8-users@googlegroups.com/msg14528.html
390 arm_control_flow_integrity = "none";