pyrox.dev
Merge pull request #245623 from tweag/fileset.toSource
File set combinators base: `lib.fileset.toSource`
authored by Silvan Mosberger and committed by GitHub d66929b7 f31b3c65
+2
.github/CODEOWNERS
+2
.github/CODEOWNERS
+1
doc/default.nix
+1
doc/default.nix
···
19
{ name = "options"; description = "NixOS / nixpkgs option handling"; }
20
{ name = "path"; description = "path functions"; }
21
{ name = "filesystem"; description = "filesystem functions"; }
22
{ name = "sources"; description = "source filtering functions"; }
23
{ name = "cli"; description = "command-line serialization functions"; }
24
{ name = "gvariant"; description = "GVariant formatted string serialization functions"; }
···
19
{ name = "options"; description = "NixOS / nixpkgs option handling"; }
20
{ name = "path"; description = "path functions"; }
21
{ name = "filesystem"; description = "filesystem functions"; }
22
+
{ name = "fileset"; description = "file set functions"; }
23
{ name = "sources"; description = "source filtering functions"; }
24
{ name = "cli"; description = "command-line serialization functions"; }
25
{ name = "gvariant"; description = "GVariant formatted string serialization functions"; }
+1
doc/functions.md
+1
doc/functions.md
+51
doc/functions/fileset.section.md
+51
doc/functions/fileset.section.md
···
···
1
+
<!-- TODO: Render this document in front of function documentation in case https://github.com/nix-community/nixdoc/issues/19 is ever supported -->
2
+
3
+
# File sets {#sec-fileset}
4
+
5
+
The [`lib.fileset`](#sec-functions-library-fileset) library allows you to work with _file sets_.
6
+
A file set is a mathematical set of local files that can be added to the Nix store for use in Nix derivations.
7
+
File sets are easy and safe to use, providing obvious and composable semantics with good error messages to prevent mistakes.
8
+
9
+
These sections apply to the entire library.
10
+
See the [function reference](#sec-functions-library-fileset) for function-specific documentation.
11
+
12
+
The file set library is currently very limited but is being expanded to include more functions over time.
13
+
14
+
## Implicit coercion from paths to file sets {#sec-fileset-path-coercion}
15
+
16
+
All functions accepting file sets as arguments can also accept [paths](https://nixos.org/manual/nix/stable/language/values.html#type-path) as arguments.
17
+
Such path arguments are implicitly coerced to file sets containing all files under that path:
18
+
- A path to a file turns into a file set containing that single file.
19
+
- A path to a directory turns into a file set containing all files _recursively_ in that directory.
20
+
21
+
If the path points to a non-existent location, an error is thrown.
22
+
23
+
::: {.note}
24
+
Just like in Git, file sets cannot represent empty directories.
25
+
Because of this, a path to a directory that contains no files (recursively) will turn into a file set containing no files.
26
+
:::
27
+
28
+
:::{.note}
29
+
File set coercion does _not_ add any of the files under the coerced paths to the store.
30
+
Only the [`toSource`](#function-library-lib.fileset.toSource) function adds files to the Nix store, and only those files contained in the `fileset` argument.
31
+
This is in contrast to using [paths in string interpolation](https://nixos.org/manual/nix/stable/language/values.html#type-path), which does add the entire referenced path to the store.
32
+
:::
33
+
34
+
### Example {#sec-fileset-path-coercion-example}
35
+
36
+
Assume we are in a local directory with a file hierarchy like this:
37
+
```
38
+
├─ a/
39
+
│ ├─ x (file)
40
+
│ └─ b/
41
+
│ └─ y (file)
42
+
└─ c/
43
+
└─ d/
44
+
```
45
+
46
+
Here's a listing of which files get included when different path expressions get coerced to file sets:
47
+
- `./.` as a file set contains both `a/x` and `a/b/y` (`c/` does not contain any files and is therefore omitted).
48
+
- `./a` as a file set contains both `a/x` and `a/b/y`.
49
+
- `./a/x` as a file set contains only `a/x`.
50
+
- `./a/b` as a file set contains only `a/b/y`.
51
+
- `./c` as a file set is empty, since neither `c` nor `c/d` contain any files.
+3
lib/README.md
+3
lib/README.md
+1
lib/default.nix
+1
lib/default.nix
+183
lib/fileset/README.md
+183
lib/fileset/README.md
···
···
1
+
# File set library
2
+
3
+
The main goal of the file set library is to be able to select local files that should be added to the Nix store.
4
+
It should have the following properties:
5
+
- Easy:
6
+
The functions should have obvious semantics, be low in number and be composable.
7
+
- Safe:
8
+
Throw early and helpful errors when mistakes are detected.
9
+
- Lazy:
10
+
Only compute values when necessary.
11
+
12
+
Non-goals are:
13
+
- Efficient:
14
+
If the abstraction proves itself worthwhile but too slow, it can be still be optimized further.
15
+
16
+
## Tests
17
+
18
+
Tests are declared in [`tests.sh`](./tests.sh) and can be run using
19
+
```
20
+
./tests.sh
21
+
```
22
+
23
+
## Benchmark
24
+
25
+
A simple benchmark against the HEAD commit can be run using
26
+
```
27
+
./benchmark.sh HEAD
28
+
```
29
+
30
+
This is intended to be run manually and is not checked by CI.
31
+
32
+
## Internal representation
33
+
34
+
The internal representation is versioned in order to allow file sets from different Nixpkgs versions to be composed with each other, see [`internal.nix`](./internal.nix) for the versions and conversions between them.
35
+
This section describes only the current representation, but past versions will have to be supported by the code.
36
+
37
+
### `fileset`
38
+
39
+
An attribute set with these values:
40
+
41
+
- `_type` (constant string `"fileset"`):
42
+
Tag to indicate this value is a file set.
43
+
44
+
- `_internalVersion` (constant string equal to the current version):
45
+
Version of the representation
46
+
47
+
- `_internalBase` (path):
48
+
Any files outside of this path cannot influence the set of files.
49
+
This is always a directory.
50
+
51
+
- `_internalTree` ([filesetTree](#filesettree)):
52
+
A tree representation of all included files under `_internalBase`.
53
+
54
+
- `__noEval` (error):
55
+
An error indicating that directly evaluating file sets is not supported.
56
+
57
+
## `filesetTree`
58
+
59
+
One of the following:
60
+
61
+
- `{ <name> = filesetTree; }`:
62
+
A directory with a nested `filesetTree` value for every directory entry.
63
+
Even entries that aren't included are present as `null` because it improves laziness and allows using this as a sort of `builtins.readDir` cache.
64
+
65
+
- `"directory"`:
66
+
A directory with all its files included recursively, allowing early cutoff for some operations.
67
+
This specific string is chosen to be compatible with `builtins.readDir` for a simpler implementation.
68
+
69
+
- `"regular"`, `"symlink"`, `"unknown"` or any other non-`"directory"` string:
70
+
A nested file with its file type.
71
+
These specific strings are chosen to be compatible with `builtins.readDir` for a simpler implementation.
72
+
Distinguishing between different file types is not strictly necessary for the functionality this library,
73
+
but it does allow nicer printing of file sets.
74
+
75
+
- `null`:
76
+
A file or directory that is excluded from the tree.
77
+
It may still exist on the file system.
78
+
79
+
## API design decisions
80
+
81
+
This section justifies API design decisions.
82
+
83
+
### Internal structure
84
+
85
+
The representation of the file set data type is internal and can be changed over time.
86
+
87
+
Arguments:
88
+
- (+) The point of this library is to provide high-level functions, users don't need to be concerned with how it's implemented
89
+
- (+) It allows adjustments to the representation, which is especially useful in the early days of the library.
90
+
- (+) It still allows the representation to be stabilized later if necessary and if it has proven itself
91
+
92
+
### Influence tracking
93
+
94
+
File set operations internally track the top-most directory that could influence the exact contents of a file set.
95
+
Specifically, `toSource` requires that the given `fileset` is completely determined by files within the directory specified by the `root` argument.
96
+
For example, even with `dir/file.txt` being the only file in `./.`, `toSource { root = ./dir; fileset = ./.; }` gives an error.
97
+
This is because `fileset` may as well be the result of filtering `./.` in a way that excludes `dir`.
98
+
99
+
Arguments:
100
+
- (+) This gives us the guarantee that adding new files to a project never breaks a file set expression.
101
+
This is also true in a lesser form for removed files:
102
+
only removing files explicitly referenced by paths can break a file set expression.
103
+
- (+) This can be removed later, if we discover it's too restrictive
104
+
- (-) It leads to errors when a sensible result could sometimes be returned, such as in the above example.
105
+
106
+
### Empty directories
107
+
108
+
File sets can only represent a _set_ of local files, directories on their own are not representable.
109
+
110
+
Arguments:
111
+
- (+) There does not seem to be a sensible set of combinators when directories can be represented on their own.
112
+
Here's some possibilities:
113
+
- `./.` represents the files in `./.` _and_ the directory itself including its subdirectories, meaning that even if there's no files, the entire structure of `./.` is preserved
114
+
115
+
In that case, what should `fileFilter (file: false) ./.` return?
116
+
It could return the entire directory structure unchanged, but with all files removed, which would not be what one would expect.
117
+
118
+
Trying to have a filter function that also supports directories will lead to the question of:
119
+
What should the behavior be if `./foo` itself is excluded but all of its contents are included?
120
+
It leads to having to define when directories are recursed into, but then we're effectively back at how the `builtins.path`-based filters work.
121
+
122
+
- `./.` represents all files in `./.` _and_ the directory itself, but not its subdirectories, meaning that at least `./.` will be preserved even if it's empty.
123
+
124
+
In that case, `intersect ./. ./foo` should only include files and no directories themselves, since `./.` includes only `./.` as a directory, and same for `./foo`, so there's no overlap in directories.
125
+
But intuitively this operation should result in the same as `./foo` – everything else is just confusing.
126
+
- (+) This matches how Git only supports files, so developers should already be used to it.
127
+
- (-) Empty directories (even if they contain nested directories) are neither representable nor preserved when coercing from paths.
128
+
- (+) It is very rare that empty directories are necessary.
129
+
- (+) We can implement a workaround, allowing `toSource` to take an extra argument for ensuring certain extra directories exist in the result.
130
+
- (-) It slows down store imports, since the evaluator needs to traverse the entire tree to remove any empty directories
131
+
- (+) This can still be optimized by introducing more Nix builtins if necessary
132
+
133
+
### String paths
134
+
135
+
File sets do not support Nix store paths in strings such as `"/nix/store/...-source"`.
136
+
137
+
Arguments:
138
+
- (+) Such paths are usually produced by derivations, which means `toSource` would either:
139
+
- Require IFD if `builtins.path` is used as the underlying primitive
140
+
- Require importing the entire `root` into the store such that derivations can be used to do the filtering
141
+
- (+) The convenient path coercion like `union ./foo ./bar` wouldn't work for absolute paths, requiring more verbose alternate interfaces:
142
+
- `let root = "/nix/store/...-source"; in union "${root}/foo" "${root}/bar"`
143
+
144
+
Verbose and dangerous because if `root` was a path, the entire path would get imported into the store.
145
+
146
+
- `toSource { root = "/nix/store/...-source"; fileset = union "./foo" "./bar"; }`
147
+
148
+
Does not allow debug printing intermediate file set contents, since we don't know the paths contents before having a `root`.
149
+
150
+
- `let fs = lib.fileset.withRoot "/nix/store/...-source"; in fs.union "./foo" "./bar"`
151
+
152
+
Makes library functions impure since they depend on the contextual root path, questionable composability.
153
+
154
+
- (+) The point of the file set abstraction is to specify which files should get imported into the store.
155
+
156
+
This use case makes little sense for files that are already in the store.
157
+
This should be a separate abstraction as e.g. `pkgs.drvLayout` instead, which could have a similar interface but be specific to derivations.
158
+
Additional capabilities could be supported that can't be done at evaluation time, such as renaming files, creating new directories, setting executable bits, etc.
159
+
160
+
### Single files
161
+
162
+
File sets cannot add single files to the store, they can only import files under directories.
163
+
164
+
Arguments:
165
+
- (+) There's no point in using this library for a single file, since you can't do anything other than add it to the store or not.
166
+
And it would be unclear how the library should behave if the one file wouldn't be added to the store:
167
+
`toSource { root = ./file.nix; fileset = <empty>; }` has no reasonable result because returing an empty store path wouldn't match the file type, and there's no way to have an empty file store path, whatever that would mean.
168
+
169
+
## To update in the future
170
+
171
+
Here's a list of places in the library that need to be updated in the future:
172
+
- > The file set library is currently very limited but is being expanded to include more functions over time.
173
+
174
+
in [the manual](../../doc/functions/fileset.section.md)
175
+
- > Currently the only way to construct file sets is using implicit coercion from paths.
176
+
177
+
in [the `toSource` reference](./default.nix)
178
+
- > For now filesets are always paths
179
+
180
+
in [the `toSource` implementation](./default.nix), also update the variable name there
181
+
- Once a tracing function exists, `__noEval` in [internal.nix](./internal.nix) should mention it
182
+
- If/Once a function to convert `lib.sources` values into file sets exists, the `_coerce` and `toSource` functions should be updated to mention that function in the error when such a value is passed
183
+
- If/Once a function exists that can optionally include a path depending on whether it exists, the error message for the path not existing in `_coerce` should mention the new function
+94
lib/fileset/benchmark.sh
+94
lib/fileset/benchmark.sh
···
···
1
+
#!/usr/bin/env bash
2
+
3
+
# Benchmarks lib.fileset
4
+
# Run:
5
+
# [nixpkgs]$ lib/fileset/benchmark.sh HEAD
6
+
7
+
set -euo pipefail
8
+
shopt -s inherit_errexit dotglob
9
+
10
+
if (( $# == 0 )); then
11
+
echo "Usage: $0 HEAD"
12
+
echo "Benchmarks the current tree against the HEAD commit. Any git ref will work."
13
+
exit 1
14
+
fi
15
+
compareTo=$1
16
+
17
+
SCRIPT_FILE=$(readlink -f "${BASH_SOURCE[0]}")
18
+
SCRIPT_DIR=$(dirname "$SCRIPT_FILE")
19
+
20
+
nixpkgs=$(cd "$SCRIPT_DIR/../.."; pwd)
21
+
22
+
tmp="$(mktemp -d)"
23
+
clean_up() {
24
+
rm -rf "$tmp"
25
+
}
26
+
trap clean_up EXIT SIGINT SIGTERM
27
+
work="$tmp/work"
28
+
mkdir "$work"
29
+
cd "$work"
30
+
31
+
# Create a fairly populated tree
32
+
touch f{0..5}
33
+
mkdir d{0..5}
34
+
mkdir e{0..5}
35
+
touch d{0..5}/f{0..5}
36
+
mkdir -p d{0..5}/d{0..5}
37
+
mkdir -p e{0..5}/e{0..5}
38
+
touch d{0..5}/d{0..5}/f{0..5}
39
+
mkdir -p d{0..5}/d{0..5}/d{0..5}
40
+
mkdir -p e{0..5}/e{0..5}/e{0..5}
41
+
touch d{0..5}/d{0..5}/d{0..5}/f{0..5}
42
+
mkdir -p d{0..5}/d{0..5}/d{0..5}/d{0..5}
43
+
mkdir -p e{0..5}/e{0..5}/e{0..5}/e{0..5}
44
+
touch d{0..5}/d{0..5}/d{0..5}/d{0..5}/f{0..5}
45
+
46
+
bench() {
47
+
NIX_PATH=nixpkgs=$1 NIX_SHOW_STATS=1 NIX_SHOW_STATS_PATH=$tmp/stats.json \
48
+
nix-instantiate --eval --strict --show-trace >/dev/null \
49
+
--expr '(import <nixpkgs/lib>).fileset.toSource { root = ./.; fileset = ./.; }'
50
+
cat "$tmp/stats.json"
51
+
}
52
+
53
+
echo "Running benchmark on index" >&2
54
+
bench "$nixpkgs" > "$tmp/new.json"
55
+
(
56
+
echo "Checking out $compareTo" >&2
57
+
git -C "$nixpkgs" worktree add --quiet "$tmp/worktree" "$compareTo"
58
+
trap 'git -C "$nixpkgs" worktree remove "$tmp/worktree"' EXIT
59
+
echo "Running benchmark on $compareTo" >&2
60
+
bench "$tmp/worktree" > "$tmp/old.json"
61
+
)
62
+
63
+
declare -a stats=(
64
+
".envs.elements"
65
+
".envs.number"
66
+
".gc.totalBytes"
67
+
".list.concats"
68
+
".list.elements"
69
+
".nrFunctionCalls"
70
+
".nrLookups"
71
+
".nrOpUpdates"
72
+
".nrPrimOpCalls"
73
+
".nrThunks"
74
+
".sets.elements"
75
+
".sets.number"
76
+
".symbols.number"
77
+
".values.number"
78
+
)
79
+
80
+
different=0
81
+
for stat in "${stats[@]}"; do
82
+
oldValue=$(jq "$stat" "$tmp/old.json")
83
+
newValue=$(jq "$stat" "$tmp/new.json")
84
+
if (( oldValue != newValue )); then
85
+
percent=$(bc <<< "scale=100; result = 100/$oldValue*$newValue; scale=4; result / 1")
86
+
if (( oldValue < newValue )); then
87
+
echo -e "Statistic $stat ($newValue) is \e[0;31m$percent% (+$(( newValue - oldValue )))\e[0m of the old value $oldValue" >&2
88
+
else
89
+
echo -e "Statistic $stat ($newValue) is \e[0;32m$percent% (-$(( oldValue - newValue )))\e[0m of the old value $oldValue" >&2
90
+
fi
91
+
(( different++ )) || true
92
+
fi
93
+
done
94
+
echo "$different stats differ between the current tree and $compareTo"
+131
lib/fileset/default.nix
+131
lib/fileset/default.nix
···
···
1
+
{ lib }:
2
+
let
3
+
4
+
inherit (import ./internal.nix { inherit lib; })
5
+
_coerce
6
+
_toSourceFilter
7
+
;
8
+
9
+
inherit (builtins)
10
+
isPath
11
+
pathExists
12
+
typeOf
13
+
;
14
+
15
+
inherit (lib.path)
16
+
hasPrefix
17
+
splitRoot
18
+
;
19
+
20
+
inherit (lib.strings)
21
+
isStringLike
22
+
;
23
+
24
+
inherit (lib.filesystem)
25
+
pathType
26
+
;
27
+
28
+
inherit (lib.sources)
29
+
cleanSourceWith
30
+
;
31
+
32
+
in {
33
+
34
+
/*
35
+
Add the local files contained in `fileset` to the store as a single [store path](https://nixos.org/manual/nix/stable/glossary#gloss-store-path) rooted at `root`.
36
+
37
+
The result is the store path as a string-like value, making it usable e.g. as the `src` of a derivation, or in string interpolation:
38
+
```nix
39
+
stdenv.mkDerivation {
40
+
src = lib.fileset.toSource { ... };
41
+
# ...
42
+
}
43
+
```
44
+
45
+
The name of the store path is always `source`.
46
+
47
+
Type:
48
+
toSource :: {
49
+
root :: Path,
50
+
fileset :: FileSet,
51
+
} -> SourceLike
52
+
53
+
Example:
54
+
# Import the current directory into the store but only include files under ./src
55
+
toSource { root = ./.; fileset = ./src; }
56
+
=> "/nix/store/...-source"
57
+
58
+
# The file set coerced from path ./bar could contain files outside the root ./foo, which is not allowed
59
+
toSource { root = ./foo; fileset = ./bar; }
60
+
=> <error>
61
+
62
+
# The root has to be a local filesystem path
63
+
toSource { root = "/nix/store/...-source"; fileset = ./.; }
64
+
=> <error>
65
+
*/
66
+
toSource = {
67
+
/*
68
+
(required) The local directory [path](https://nixos.org/manual/nix/stable/language/values.html#type-path) that will correspond to the root of the resulting store path.
69
+
Paths in [strings](https://nixos.org/manual/nix/stable/language/values.html#type-string), including Nix store paths, cannot be passed as `root`.
70
+
`root` has to be a directory.
71
+
72
+
<!-- Ignore the indentation here, this is a nixdoc rendering bug that needs to be fixed -->
73
+
:::{.note}
74
+
Changing `root` only affects the directory structure of the resulting store path, it does not change which files are added to the store.
75
+
The only way to change which files get added to the store is by changing the `fileset` attribute.
76
+
:::
77
+
*/
78
+
root,
79
+
/*
80
+
(required) The file set whose files to import into the store.
81
+
Currently the only way to construct file sets is using [implicit coercion from paths](#sec-fileset-path-coercion).
82
+
If a directory does not recursively contain any file, it is omitted from the store path contents.
83
+
*/
84
+
fileset,
85
+
}:
86
+
let
87
+
# We cannot rename matched attribute arguments, so let's work around it with an extra `let in` statement
88
+
# For now filesets are always paths
89
+
filesetPath = fileset;
90
+
in
91
+
let
92
+
fileset = _coerce "lib.fileset.toSource: `fileset`" filesetPath;
93
+
rootFilesystemRoot = (splitRoot root).root;
94
+
filesetFilesystemRoot = (splitRoot fileset._internalBase).root;
95
+
in
96
+
if ! isPath root then
97
+
if isStringLike root then
98
+
throw ''
99
+
lib.fileset.toSource: `root` "${toString root}" is a string-like value, but it should be a path instead.
100
+
Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.''
101
+
else
102
+
throw ''
103
+
lib.fileset.toSource: `root` is of type ${typeOf root}, but it should be a path instead.''
104
+
# Currently all Nix paths have the same filesystem root, but this could change in the future.
105
+
# See also ../path/README.md
106
+
else if rootFilesystemRoot != filesetFilesystemRoot then
107
+
throw ''
108
+
lib.fileset.toSource: Filesystem roots are not the same for `fileset` and `root` "${toString root}":
109
+
`root`: root "${toString rootFilesystemRoot}"
110
+
`fileset`: root "${toString filesetFilesystemRoot}"
111
+
Different roots are not supported.''
112
+
else if ! pathExists root then
113
+
throw ''
114
+
lib.fileset.toSource: `root` ${toString root} does not exist.''
115
+
else if pathType root != "directory" then
116
+
throw ''
117
+
lib.fileset.toSource: `root` ${toString root} is a file, but it should be a directory instead. Potential solutions:
118
+
- If you want to import the file into the store _without_ a containing directory, use string interpolation or `builtins.path` instead of this function.
119
+
- If you want to import the file into the store _with_ a containing directory, set `root` to the containing directory, such as ${toString (dirOf root)}, and set `fileset` to the file path.''
120
+
else if ! hasPrefix root fileset._internalBase then
121
+
throw ''
122
+
lib.fileset.toSource: `fileset` could contain files in ${toString fileset._internalBase}, which is not under the `root` ${toString root}. Potential solutions:
123
+
- Set `root` to ${toString fileset._internalBase} or any directory higher up. This changes the layout of the resulting store path.
124
+
- Set `fileset` to a file set that cannot contain files outside the `root` ${toString root}. This could change the files included in the result.''
125
+
else
126
+
cleanSourceWith {
127
+
name = "source";
128
+
src = root;
129
+
filter = _toSourceFilter fileset;
130
+
};
131
+
}
+274
lib/fileset/internal.nix
+274
lib/fileset/internal.nix
···
···
1
+
{ lib ? import ../. }:
2
+
let
3
+
4
+
inherit (builtins)
5
+
isAttrs
6
+
isPath
7
+
isString
8
+
pathExists
9
+
readDir
10
+
typeOf
11
+
split
12
+
;
13
+
14
+
inherit (lib.attrsets)
15
+
attrValues
16
+
mapAttrs
17
+
;
18
+
19
+
inherit (lib.filesystem)
20
+
pathType
21
+
;
22
+
23
+
inherit (lib.lists)
24
+
all
25
+
elemAt
26
+
length
27
+
;
28
+
29
+
inherit (lib.path)
30
+
append
31
+
splitRoot
32
+
;
33
+
34
+
inherit (lib.path.subpath)
35
+
components
36
+
;
37
+
38
+
inherit (lib.strings)
39
+
isStringLike
40
+
concatStringsSep
41
+
substring
42
+
stringLength
43
+
;
44
+
45
+
in
46
+
# Rare case of justified usage of rec:
47
+
# - This file is internal, so the return value doesn't matter, no need to make things overridable
48
+
# - The functions depend on each other
49
+
# - We want to expose all of these functions for easy testing
50
+
rec {
51
+
52
+
# If you change the internal representation, make sure to:
53
+
# - Update this version
54
+
# - Adjust _coerce to also accept and coerce older versions
55
+
# - Update the description of the internal representation in ./README.md
56
+
_currentVersion = 0;
57
+
58
+
# Create a fileset, see ./README.md#fileset
59
+
# Type: path -> filesetTree -> fileset
60
+
_create = base: tree: {
61
+
_type = "fileset";
62
+
63
+
_internalVersion = _currentVersion;
64
+
_internalBase = base;
65
+
_internalTree = tree;
66
+
67
+
# Double __ to make it be evaluated and ordered first
68
+
__noEval = throw ''
69
+
lib.fileset: Directly evaluating a file set is not supported. Use `lib.fileset.toSource` to turn it into a usable source instead.'';
70
+
};
71
+
72
+
# Coerce a value to a fileset, erroring when the value cannot be coerced.
73
+
# The string gives the context for error messages.
74
+
# Type: String -> Path -> fileset
75
+
_coerce = context: value:
76
+
if value._type or "" == "fileset" then
77
+
if value._internalVersion > _currentVersion then
78
+
throw ''
79
+
${context} is a file set created from a future version of the file set library with a different internal representation:
80
+
- Internal version of the file set: ${toString value._internalVersion}
81
+
- Internal version of the library: ${toString _currentVersion}
82
+
Make sure to update your Nixpkgs to have a newer version of `lib.fileset`.''
83
+
else
84
+
value
85
+
else if ! isPath value then
86
+
if isStringLike value then
87
+
throw ''
88
+
${context} "${toString value}" is a string-like value, but it should be a path instead.
89
+
Paths represented as strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.''
90
+
else
91
+
throw ''
92
+
${context} is of type ${typeOf value}, but it should be a path instead.''
93
+
else if ! pathExists value then
94
+
throw ''
95
+
${context} ${toString value} does not exist.''
96
+
else
97
+
_singleton value;
98
+
99
+
# Create a file set from a path.
100
+
# Type: Path -> fileset
101
+
_singleton = path:
102
+
let
103
+
type = pathType path;
104
+
in
105
+
if type == "directory" then
106
+
_create path type
107
+
else
108
+
# This turns a file path ./default.nix into a fileset with
109
+
# - _internalBase: ./.
110
+
# - _internalTree: {
111
+
# "default.nix" = <type>;
112
+
# # Other directory entries
113
+
# <name> = null;
114
+
# }
115
+
# See ./README.md#single-files
116
+
_create (dirOf path)
117
+
(_nestTree
118
+
(dirOf path)
119
+
[ (baseNameOf path) ]
120
+
type
121
+
);
122
+
123
+
/*
124
+
Nest a filesetTree under some extra components, while filling out all the other directory entries that aren't included with null
125
+
126
+
_nestTree ./. [ "foo" "bar" ] tree == {
127
+
foo = {
128
+
bar = tree;
129
+
<other-entries> = null;
130
+
}
131
+
<other-entries> = null;
132
+
}
133
+
134
+
Type: Path -> [ String ] -> filesetTree -> filesetTree
135
+
*/
136
+
_nestTree = targetBase: extraComponents: tree:
137
+
let
138
+
recurse = index: focusPath:
139
+
if index == length extraComponents then
140
+
tree
141
+
else
142
+
mapAttrs (_: _: null) (readDir focusPath)
143
+
// {
144
+
${elemAt extraComponents index} = recurse (index + 1) (append focusPath (elemAt extraComponents index));
145
+
};
146
+
in
147
+
recurse 0 targetBase;
148
+
149
+
# Expand "directory" filesetTree representation to the equivalent { <name> = filesetTree; }
150
+
# Type: Path -> filesetTree -> { <name> = filesetTree; }
151
+
_directoryEntries = path: value:
152
+
if isAttrs value then
153
+
value
154
+
else
155
+
readDir path;
156
+
157
+
/*
158
+
Simplify a filesetTree recursively:
159
+
- Replace all directories that have no files with `null`
160
+
This removes directories that would be empty
161
+
- Replace all directories with all files with `"directory"`
162
+
This speeds up the source filter function
163
+
164
+
Note that this function is strict, it evaluates the entire tree
165
+
166
+
Type: Path -> filesetTree -> filesetTree
167
+
*/
168
+
_simplifyTree = path: tree:
169
+
if tree == "directory" || isAttrs tree then
170
+
let
171
+
entries = _directoryEntries path tree;
172
+
simpleSubtrees = mapAttrs (name: _simplifyTree (path + "/${name}")) entries;
173
+
subtreeValues = attrValues simpleSubtrees;
174
+
in
175
+
# This triggers either when all files in a directory are filtered out
176
+
# Or when the directory doesn't contain any files at all
177
+
if all isNull subtreeValues then
178
+
null
179
+
# Triggers when we have the same as a `readDir path`, so we can turn it back into an equivalent "directory".
180
+
else if all isString subtreeValues then
181
+
"directory"
182
+
else
183
+
simpleSubtrees
184
+
else
185
+
tree;
186
+
187
+
# Turn a fileset into a source filter function suitable for `builtins.path`
188
+
# Only directories recursively containing at least one files are recursed into
189
+
# Type: Path -> fileset -> (String -> String -> Bool)
190
+
_toSourceFilter = fileset:
191
+
let
192
+
# Simplify the tree, necessary to make sure all empty directories are null
193
+
# which has the effect that they aren't included in the result
194
+
tree = _simplifyTree fileset._internalBase fileset._internalTree;
195
+
196
+
# Decompose the base into its components
197
+
# See ../path/README.md for why we're not just using `toString`
198
+
baseComponents = components (splitRoot fileset._internalBase).subpath;
199
+
200
+
# The base path as a string with a single trailing slash
201
+
baseString =
202
+
if baseComponents == [] then
203
+
# Need to handle the filesystem root specially
204
+
"/"
205
+
else
206
+
"/" + concatStringsSep "/" baseComponents + "/";
207
+
208
+
baseLength = stringLength baseString;
209
+
210
+
# Check whether a list of path components under the base path exists in the tree.
211
+
# This function is called often, so it should be fast.
212
+
# Type: [ String ] -> Bool
213
+
inTree = components:
214
+
let
215
+
recurse = index: localTree:
216
+
if isAttrs localTree then
217
+
# We have an attribute set, meaning this is a directory with at least one file
218
+
if index >= length components then
219
+
# The path may have no more components though, meaning the filter is running on the directory itself,
220
+
# so we always include it, again because there's at least one file in it.
221
+
true
222
+
else
223
+
# If we do have more components, the filter runs on some entry inside this directory, so we need to recurse
224
+
# We do +2 because builtins.split is an interleaved list of the inbetweens and the matches
225
+
recurse (index + 2) localTree.${elemAt components index}
226
+
else
227
+
# If it's not an attribute set it can only be either null (in which case it's not included)
228
+
# or a string ("directory" or "regular", etc.) in which case it's included
229
+
localTree != null;
230
+
in recurse 0 tree;
231
+
232
+
# Filter suited when there's no files
233
+
empty = _: _: false;
234
+
235
+
# Filter suited when there's some files
236
+
# This can't be used for when there's no files, because the base directory is always included
237
+
nonEmpty =
238
+
path: _:
239
+
let
240
+
# Add a slash to the path string, turning "/foo" to "/foo/",
241
+
# making sure to not have any false prefix matches below.
242
+
# Note that this would produce "//" for "/",
243
+
# but builtins.path doesn't call the filter function on the `path` argument itself,
244
+
# meaning this function can never receive "/" as an argument
245
+
pathSlash = path + "/";
246
+
in
247
+
# Same as `hasPrefix pathSlash baseString`, but more efficient.
248
+
# With base /foo/bar we need to include /foo:
249
+
# hasPrefix "/foo/" "/foo/bar/"
250
+
if substring 0 (stringLength pathSlash) baseString == pathSlash then
251
+
true
252
+
# Same as `! hasPrefix baseString pathSlash`, but more efficient.
253
+
# With base /foo/bar we need to exclude /baz
254
+
# ! hasPrefix "/baz/" "/foo/bar/"
255
+
else if substring 0 baseLength pathSlash != baseString then
256
+
false
257
+
else
258
+
# Same as `removePrefix baseString path`, but more efficient.
259
+
# From the above code we know that hasPrefix baseString pathSlash holds, so this is safe.
260
+
# We don't use pathSlash here because we only needed the trailing slash for the prefix matching.
261
+
# With base /foo and path /foo/bar/baz this gives
262
+
# inTree (split "/" (removePrefix "/foo/" "/foo/bar/baz"))
263
+
# == inTree (split "/" "bar/baz")
264
+
# == inTree [ "bar" "baz" ]
265
+
inTree (split "/" (substring baseLength (-1) path));
266
+
in
267
+
# Special case because the code below assumes that the _internalBase is always included in the result
268
+
# which shouldn't be done when we have no files at all in the base
269
+
if tree == null then
270
+
empty
271
+
else
272
+
nonEmpty;
273
+
274
+
}
+26
lib/fileset/mock-splitRoot.nix
+26
lib/fileset/mock-splitRoot.nix
···
···
1
+
# This overlay implements mocking of the lib.path.splitRoot function
2
+
# It pretends that the last component named "mock-root" is the root:
3
+
#
4
+
# splitRoot /foo/mock-root/bar/mock-root/baz
5
+
# => {
6
+
# root = /foo/mock-root/bar/mock-root;
7
+
# subpath = "./baz";
8
+
# }
9
+
self: super: {
10
+
path = super.path // {
11
+
splitRoot = path:
12
+
let
13
+
parts = super.path.splitRoot path;
14
+
components = self.path.subpath.components parts.subpath;
15
+
count = self.length components;
16
+
rootIndex = count - self.lists.findFirstIndex
17
+
(component: component == "mock-root")
18
+
(self.length components)
19
+
(self.reverseList components);
20
+
root = self.path.append parts.root (self.path.subpath.join (self.take rootIndex components));
21
+
subpath = self.path.subpath.join (self.drop rootIndex components);
22
+
in {
23
+
inherit root subpath;
24
+
};
25
+
};
26
+
}
+350
lib/fileset/tests.sh
+350
lib/fileset/tests.sh
···
···
1
+
#!/usr/bin/env bash
2
+
3
+
# Tests lib.fileset
4
+
# Run:
5
+
# [nixpkgs]$ lib/fileset/tests.sh
6
+
# or:
7
+
# [nixpkgs]$ nix-build lib/tests/release.nix
8
+
9
+
set -euo pipefail
10
+
shopt -s inherit_errexit dotglob
11
+
12
+
die() {
13
+
# The second to last entry contains the line number of the top-level caller
14
+
lineIndex=$(( ${#BASH_LINENO[@]} - 2 ))
15
+
echo >&2 -e "test case at ${BASH_SOURCE[0]}:${BASH_LINENO[$lineIndex]} failed:" "$@"
16
+
exit 1
17
+
}
18
+
19
+
if test -n "${TEST_LIB:-}"; then
20
+
NIX_PATH=nixpkgs="$(dirname "$TEST_LIB")"
21
+
else
22
+
NIX_PATH=nixpkgs="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.."; pwd)"
23
+
fi
24
+
export NIX_PATH
25
+
26
+
tmp="$(mktemp -d)"
27
+
clean_up() {
28
+
rm -rf "$tmp"
29
+
}
30
+
trap clean_up EXIT SIGINT SIGTERM
31
+
work="$tmp/work"
32
+
mkdir "$work"
33
+
cd "$work"
34
+
35
+
# Crudely unquotes a JSON string by just taking everything between the first and the second quote.
36
+
# We're only using this for resulting /nix/store paths, which can't contain " anyways,
37
+
# nor can they contain any other characters that would need to be escaped specially in JSON
38
+
# This way we don't need to add a dependency on e.g. jq
39
+
crudeUnquoteJSON() {
40
+
cut -d \" -f2
41
+
}
42
+
43
+
prefixExpression='let
44
+
lib = import <nixpkgs/lib>;
45
+
internal = import <nixpkgs/lib/fileset/internal.nix> {
46
+
inherit lib;
47
+
};
48
+
in
49
+
with lib;
50
+
with internal;
51
+
with lib.fileset;'
52
+
53
+
# Check that a nix expression evaluates successfully (strictly, coercing to json, read-write-mode).
54
+
# The expression has `lib.fileset` in scope.
55
+
# If a second argument is provided, the result is checked against it as a regex.
56
+
# Otherwise, the result is output.
57
+
# Usage: expectSuccess NIX [REGEX]
58
+
expectSuccess() {
59
+
local expr=$1
60
+
if [[ "$#" -gt 1 ]]; then
61
+
local expectedResultRegex=$2
62
+
fi
63
+
if ! result=$(nix-instantiate --eval --strict --json --read-write-mode --show-trace \
64
+
--expr "$prefixExpression $expr"); then
65
+
die "$expr failed to evaluate, but it was expected to succeed"
66
+
fi
67
+
if [[ -v expectedResultRegex ]]; then
68
+
if [[ ! "$result" =~ $expectedResultRegex ]]; then
69
+
die "$expr should have evaluated to this regex pattern:\n\n$expectedResultRegex\n\nbut this was the actual result:\n\n$result"
70
+
fi
71
+
else
72
+
echo "$result"
73
+
fi
74
+
}
75
+
76
+
# Check that a nix expression fails to evaluate (strictly, coercing to json, read-write-mode).
77
+
# And check the received stderr against a regex
78
+
# The expression has `lib.fileset` in scope.
79
+
# Usage: expectFailure NIX REGEX
80
+
expectFailure() {
81
+
local expr=$1
82
+
local expectedErrorRegex=$2
83
+
if result=$(nix-instantiate --eval --strict --json --read-write-mode --show-trace 2>"$tmp/stderr" \
84
+
--expr "$prefixExpression $expr"); then
85
+
die "$expr evaluated successfully to $result, but it was expected to fail"
86
+
fi
87
+
stderr=$(<"$tmp/stderr")
88
+
if [[ ! "$stderr" =~ $expectedErrorRegex ]]; then
89
+
die "$expr should have errored with this regex pattern:\n\n$expectedErrorRegex\n\nbut this was the actual error:\n\n$stderr"
90
+
fi
91
+
}
92
+
93
+
# We conditionally use inotifywait in checkFileset.
94
+
# Check early whether it's available
95
+
# TODO: Darwin support, though not crucial since we have Linux CI
96
+
if type inotifywait 2>/dev/null >/dev/null; then
97
+
canMonitorFiles=1
98
+
else
99
+
echo "Warning: Not checking that excluded files don't get accessed since inotifywait is not available" >&2
100
+
canMonitorFiles=
101
+
fi
102
+
103
+
# Check whether a file set includes/excludes declared paths as expected, usage:
104
+
#
105
+
# tree=(
106
+
# [a/b] =1 # Declare that file a/b should exist and expect it to be included in the store path
107
+
# [c/a] = # Declare that file c/a should exist and expect it to be excluded in the store path
108
+
# [c/d/]= # Declare that directory c/d/ should exist and expect it to be excluded in the store path
109
+
# )
110
+
# checkFileset './a' # Pass the fileset as the argument
111
+
declare -A tree
112
+
checkFileset() (
113
+
# New subshell so that we can have a separate trap handler, see `trap` below
114
+
local fileset=$1
115
+
116
+
# Process the tree into separate arrays for included paths, excluded paths and excluded files.
117
+
# Also create all the paths in the local directory
118
+
local -a included=()
119
+
local -a excluded=()
120
+
local -a excludedFiles=()
121
+
for p in "${!tree[@]}"; do
122
+
# If keys end with a `/` we treat them as directories, otherwise files
123
+
if [[ "$p" =~ /$ ]]; then
124
+
mkdir -p "$p"
125
+
isFile=
126
+
else
127
+
mkdir -p "$(dirname "$p")"
128
+
touch "$p"
129
+
isFile=1
130
+
fi
131
+
case "${tree[$p]}" in
132
+
1)
133
+
included+=("$p")
134
+
;;
135
+
0)
136
+
excluded+=("$p")
137
+
if [[ -n "$isFile" ]]; then
138
+
excludedFiles+=("$p")
139
+
fi
140
+
;;
141
+
*)
142
+
die "Unsupported tree value: ${tree[$p]}"
143
+
esac
144
+
done
145
+
146
+
# Start inotifywait in the background to monitor all excluded files (if any)
147
+
if [[ -n "$canMonitorFiles" ]] && (( "${#excludedFiles[@]}" != 0 )); then
148
+
coproc watcher {
149
+
# inotifywait outputs a string on stderr when ready
150
+
# Redirect it to stdout so we can access it from the coproc's stdout fd
151
+
# exec so that the coprocess is inotify itself, making the kill below work correctly
152
+
# See below why we listen to both open and delete_self events
153
+
exec inotifywait --format='%e %w' --event open,delete_self --monitor "${excludedFiles[@]}" 2>&1
154
+
}
155
+
# This will trigger when this subshell exits, no matter if successful or not
156
+
# After exiting the subshell, the parent shell will continue executing
157
+
trap 'kill "${watcher_PID}"' exit
158
+
159
+
# Synchronously wait until inotifywait is ready
160
+
while read -r -u "${watcher[0]}" line && [[ "$line" != "Watches established." ]]; do
161
+
:
162
+
done
163
+
fi
164
+
165
+
# Call toSource with the fileset, triggering open events for all files that are added to the store
166
+
expression="toSource { root = ./.; fileset = $fileset; }"
167
+
# crudeUnquoteJSON is safe because we get back a store path in a string
168
+
storePath=$(expectSuccess "$expression" | crudeUnquoteJSON)
169
+
170
+
# Remove all files immediately after, triggering delete_self events for all of them
171
+
rm -rf -- *
172
+
173
+
# Only check for the inotify events if we actually started inotify earlier
174
+
if [[ -v watcher ]]; then
175
+
# Get the first event
176
+
read -r -u "${watcher[0]}" event file
177
+
178
+
# There's only these two possible event timelines:
179
+
# - open, ..., open, delete_self, ..., delete_self: If some excluded files were read
180
+
# - delete_self, ..., delete_self: If no excluded files were read
181
+
# So by looking at the first event we can figure out which one it is!
182
+
case "$event" in
183
+
OPEN)
184
+
die "$expression opened excluded file $file when it shouldn't have"
185
+
;;
186
+
DELETE_SELF)
187
+
# Expected events
188
+
;;
189
+
*)
190
+
die "Unexpected event type '$event' on file $file that should be excluded"
191
+
;;
192
+
esac
193
+
fi
194
+
195
+
# For each path that should be included, make sure it does occur in the resulting store path
196
+
for p in "${included[@]}"; do
197
+
if [[ ! -e "$storePath/$p" ]]; then
198
+
die "$expression doesn't include path $p when it should have"
199
+
fi
200
+
done
201
+
202
+
# For each path that should be excluded, make sure it doesn't occur in the resulting store path
203
+
for p in "${excluded[@]}"; do
204
+
if [[ -e "$storePath/$p" ]]; then
205
+
die "$expression included path $p when it shouldn't have"
206
+
fi
207
+
done
208
+
)
209
+
210
+
211
+
#### Error messages #####
212
+
213
+
# Absolute paths in strings cannot be passed as `root`
214
+
expectFailure 'toSource { root = "/nix/store/foobar"; fileset = ./.; }' 'lib.fileset.toSource: `root` "/nix/store/foobar" is a string-like value, but it should be a path instead.
215
+
\s*Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.'
216
+
217
+
# Only paths are accepted as `root`
218
+
expectFailure 'toSource { root = 10; fileset = ./.; }' 'lib.fileset.toSource: `root` is of type int, but it should be a path instead.'
219
+
220
+
# Different filesystem roots in root and fileset are not supported
221
+
mkdir -p {foo,bar}/mock-root
222
+
expectFailure 'with ((import <nixpkgs/lib>).extend (import <nixpkgs/lib/fileset/mock-splitRoot.nix>)).fileset;
223
+
toSource { root = ./foo/mock-root; fileset = ./bar/mock-root; }
224
+
' 'lib.fileset.toSource: Filesystem roots are not the same for `fileset` and `root` "'"$work"'/foo/mock-root":
225
+
\s*`root`: root "'"$work"'/foo/mock-root"
226
+
\s*`fileset`: root "'"$work"'/bar/mock-root"
227
+
\s*Different roots are not supported.'
228
+
rm -rf *
229
+
230
+
# `root` needs to exist
231
+
expectFailure 'toSource { root = ./a; fileset = ./.; }' 'lib.fileset.toSource: `root` '"$work"'/a does not exist.'
232
+
233
+
# `root` needs to be a file
234
+
touch a
235
+
expectFailure 'toSource { root = ./a; fileset = ./a; }' 'lib.fileset.toSource: `root` '"$work"'/a is a file, but it should be a directory instead. Potential solutions:
236
+
\s*- If you want to import the file into the store _without_ a containing directory, use string interpolation or `builtins.path` instead of this function.
237
+
\s*- If you want to import the file into the store _with_ a containing directory, set `root` to the containing directory, such as '"$work"', and set `fileset` to the file path.'
238
+
rm -rf *
239
+
240
+
# Only paths under `root` should be able to influence the result
241
+
mkdir a
242
+
expectFailure 'toSource { root = ./a; fileset = ./.; }' 'lib.fileset.toSource: `fileset` could contain files in '"$work"', which is not under the `root` '"$work"'/a. Potential solutions:
243
+
\s*- Set `root` to '"$work"' or any directory higher up. This changes the layout of the resulting store path.
244
+
\s*- Set `fileset` to a file set that cannot contain files outside the `root` '"$work"'/a. This could change the files included in the result.'
245
+
rm -rf *
246
+
247
+
# Path coercion only works for paths
248
+
expectFailure 'toSource { root = ./.; fileset = 10; }' 'lib.fileset.toSource: `fileset` is of type int, but it should be a path instead.'
249
+
expectFailure 'toSource { root = ./.; fileset = "/some/path"; }' 'lib.fileset.toSource: `fileset` "/some/path" is a string-like value, but it should be a path instead.
250
+
\s*Paths represented as strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.'
251
+
252
+
# Path coercion errors for non-existent paths
253
+
expectFailure 'toSource { root = ./.; fileset = ./a; }' 'lib.fileset.toSource: `fileset` '"$work"'/a does not exist.'
254
+
255
+
# File sets cannot be evaluated directly
256
+
expectFailure '_create ./. null' 'lib.fileset: Directly evaluating a file set is not supported. Use `lib.fileset.toSource` to turn it into a usable source instead.'
257
+
258
+
# Future versions of the internal representation are unsupported
259
+
expectFailure '_coerce "<tests>: value" { _type = "fileset"; _internalVersion = 1; }' '<tests>: value is a file set created from a future version of the file set library with a different internal representation:
260
+
\s*- Internal version of the file set: 1
261
+
\s*- Internal version of the library: 0
262
+
\s*Make sure to update your Nixpkgs to have a newer version of `lib.fileset`.'
263
+
264
+
# _create followed by _coerce should give the inputs back without any validation
265
+
expectSuccess '{
266
+
inherit (_coerce "<test>" (_create "base" "tree"))
267
+
_internalVersion _internalBase _internalTree;
268
+
}' '\{"_internalBase":"base","_internalTree":"tree","_internalVersion":0\}'
269
+
270
+
#### Resulting store path ####
271
+
272
+
# The store path name should be "source"
273
+
expectSuccess 'toSource { root = ./.; fileset = ./.; }' '"'"${NIX_STORE_DIR:-/nix/store}"'/.*-source"'
274
+
275
+
# We should be able to import an empty directory and end up with an empty result
276
+
tree=(
277
+
)
278
+
checkFileset './.'
279
+
280
+
# Directories recursively containing no files are not included
281
+
tree=(
282
+
[e/]=0
283
+
[d/e/]=0
284
+
[d/d/e/]=0
285
+
[d/d/f]=1
286
+
[d/f]=1
287
+
[f]=1
288
+
)
289
+
checkFileset './.'
290
+
291
+
# Check trees that could cause a naïve string prefix checking implementation to fail
292
+
tree=(
293
+
[a]=0
294
+
[ab/x]=0
295
+
[ab/xy]=1
296
+
[ab/xyz]=0
297
+
[abc]=0
298
+
)
299
+
checkFileset './ab/xy'
300
+
301
+
# Check path coercion examples in ../../doc/functions/fileset.section.md
302
+
tree=(
303
+
[a/x]=1
304
+
[a/b/y]=1
305
+
[c/]=0
306
+
[c/d/]=0
307
+
)
308
+
checkFileset './.'
309
+
310
+
tree=(
311
+
[a/x]=1
312
+
[a/b/y]=1
313
+
[c/]=0
314
+
[c/d/]=0
315
+
)
316
+
checkFileset './a'
317
+
318
+
tree=(
319
+
[a/x]=1
320
+
[a/b/y]=0
321
+
[c/]=0
322
+
[c/d/]=0
323
+
)
324
+
checkFileset './a/x'
325
+
326
+
tree=(
327
+
[a/x]=0
328
+
[a/b/y]=1
329
+
[c/]=0
330
+
[c/d/]=0
331
+
)
332
+
checkFileset './a/b'
333
+
334
+
tree=(
335
+
[a/x]=0
336
+
[a/b/y]=0
337
+
[c/]=0
338
+
[c/d/]=0
339
+
)
340
+
checkFileset './c'
341
+
342
+
# Test the source filter for the somewhat special case of files in the filesystem root
343
+
# We can't easily test this with the above functions because we can't write to the filesystem root and we don't want to make any assumptions which files are there in the sandbox
344
+
expectSuccess '_toSourceFilter (_create /. null) "/foo" ""' 'false'
345
+
expectSuccess '_toSourceFilter (_create /. { foo = "regular"; }) "/foo" ""' 'true'
346
+
expectSuccess '_toSourceFilter (_create /. { foo = null; }) "/foo" ""' 'false'
347
+
348
+
# TODO: Once we have combinators and a property testing library, derive property tests from https://en.wikipedia.org/wiki/Algebra_of_sets
349
+
350
+
echo >&2 tests ok
+5
-1
lib/tests/release.nix
+5
-1
lib/tests/release.nix
···
6
}:
7
8
let
9
testWithNix = nix:
10
pkgs.runCommand "nixpkgs-lib-tests-nix-${nix.version}" {
11
buildInputs = [
···
24
];
25
nativeBuildInputs = [
26
nix
27
-
];
28
strictDeps = true;
29
} ''
30
datadir="${nix}/share"
···
49
50
echo "Running lib/tests/sources.sh"
51
TEST_LIB=$PWD/lib bash lib/tests/sources.sh
52
53
echo "Running lib/tests/systems.nix"
54
[[ $(nix-instantiate --eval --strict lib/tests/systems.nix | tee /dev/stderr) == '[ ]' ]];
···
6
}:
7
8
let
9
+
lib = import ../.;
10
testWithNix = nix:
11
pkgs.runCommand "nixpkgs-lib-tests-nix-${nix.version}" {
12
buildInputs = [
···
25
];
26
nativeBuildInputs = [
27
nix
28
+
] ++ lib.optional pkgs.stdenv.isLinux pkgs.inotify-tools;
29
strictDeps = true;
30
} ''
31
datadir="${nix}/share"
···
50
51
echo "Running lib/tests/sources.sh"
52
TEST_LIB=$PWD/lib bash lib/tests/sources.sh
53
+
54
+
echo "Running lib/fileset/tests.sh"
55
+
TEST_LIB=$PWD/lib bash lib/fileset/tests.sh
56
57
echo "Running lib/tests/systems.nix"
58
[[ $(nix-instantiate --eval --strict lib/tests/systems.nix | tee /dev/stderr) == '[ ]' ]];