···58585959- `_internalBase` (path):
6060 Any files outside of this path cannot influence the set of files.
6161- This is always a directory.
6161+ This is always a directory and should be as long as possible.
6262+ This is used by `lib.fileset.toSource` to check that all files are under the `root` argument
62636364- `_internalBaseRoot` (path):
6465 The filesystem root of `_internalBase`, same as `(lib.path.splitRoot _internalBase).root`.
···143144 - (-) Leaves us with no identity element for `union` and no reasonable return value for `unions []`.
144145 From a set theory perspective, which has a well-known notion of empty sets, this is unintuitive.
145146147147+### No intersection for lists
148148+149149+While there is `intersection a b`, there is no function `intersections [ a b c ]`.
150150+151151+Arguments:
152152+- (+) There is no known use case for such a function, it can be added later if a use case arises
153153+- (+) There is no suitable return value for `intersections [ ]`, see also "Nullary intersections" [here](https://en.wikipedia.org/w/index.php?title=List_of_set_identities_and_relations&oldid=1177174035#Definitions)
154154+ - (-) Could throw an error for that case
155155+ - (-) Create a special value to represent "all the files" and return that
156156+ - (+) Such a value could then not be used with `fileFilter` unless the internal representation is changed considerably
157157+ - (-) Could return the empty file set
158158+ - (+) This would be wrong in set theory
159159+- (-) Inconsistent with `union` and `unions`
160160+161161+### Intersection base path
162162+163163+The base path of the result of an `intersection` is the longest base path of the arguments.
164164+E.g. the base path of `intersection ./foo ./foo/bar` is `./foo/bar`.
165165+Meanwhile `intersection ./foo ./bar` returns the empty file set without a base path.
166166+167167+Arguments:
168168+- Alternative: Use the common prefix of all base paths as the resulting base path
169169+ - (-) This is unnecessarily strict, because the purpose of the base path is to track the directory under which files _could_ be in the file set. It should be as long as possible.
170170+ All files contained in `intersection ./foo ./foo/bar` will be under `./foo/bar` (never just under `./foo`), and `intersection ./foo ./bar` will never contain any files (never under `./.`).
171171+ This would lead to `toSource` having to unexpectedly throw errors for cases such as `toSource { root = ./foo; fileset = intersect ./foo base; }`, where `base` may be `./bar` or `./.`.
172172+ - (-) There is no benefit to the user, since base path is not directly exposed in the interface
173173+146174### Empty directories
147175148148-File sets can only represent a _set_ of local files, directories on their own are not representable.
176176+File sets can only represent a _set_ of local files.
177177+Directories on their own are not representable.
149178150179Arguments:
151180- (+) There does not seem to be a sensible set of combinators when directories can be represented on their own.
···161190162191 - `./.` represents all files in `./.` _and_ the directory itself, but not its subdirectories, meaning that at least `./.` will be preserved even if it's empty.
163192164164- In that case, `intersect ./. ./foo` should only include files and no directories themselves, since `./.` includes only `./.` as a directory, and same for `./foo`, so there's no overlap in directories.
193193+ In that case, `intersection ./. ./foo` should only include files and no directories themselves, since `./.` includes only `./.` as a directory, and same for `./foo`, so there's no overlap in directories.
165194 But intuitively this operation should result in the same as `./foo` – everything else is just confusing.
166195- (+) This matches how Git only supports files, so developers should already be used to it.
167196- (-) Empty directories (even if they contain nested directories) are neither representable nor preserved when coercing from paths.
+41
lib/fileset/default.nix
···77 _toSourceFilter
88 _unionMany
99 _printFileset
1010+ _intersection
1011 ;
11121213 inherit (builtins)
···1819 ;
19202021 inherit (lib.lists)
2222+ elemAt
2123 imap0
2224 ;
2325···275277 (_coerceMany "lib.fileset.unions")
276278 _unionMany
277279 ];
280280+281281+ /*
282282+ The file set containing all files that are in both of two given file sets.
283283+ See also [Intersection (set theory)](https://en.wikipedia.org/wiki/Intersection_(set_theory)).
284284+285285+ The given file sets are evaluated as lazily as possible,
286286+ with the first argument being evaluated first if needed.
287287+288288+ Type:
289289+ intersection :: FileSet -> FileSet -> FileSet
290290+291291+ Example:
292292+ # Limit the selected files to the ones in ./., so only ./src and ./Makefile
293293+ intersection ./. (unions [ ../LICENSE ./src ./Makefile ])
294294+ */
295295+ intersection =
296296+ # The first file set.
297297+ # This argument can also be a path,
298298+ # which gets [implicitly coerced to a file set](#sec-fileset-path-coercion).
299299+ fileset1:
300300+ # The second file set.
301301+ # This argument can also be a path,
302302+ # which gets [implicitly coerced to a file set](#sec-fileset-path-coercion).
303303+ fileset2:
304304+ let
305305+ filesets = _coerceMany "lib.fileset.intersection" [
306306+ {
307307+ context = "first argument";
308308+ value = fileset1;
309309+ }
310310+ {
311311+ context = "second argument";
312312+ value = fileset2;
313313+ }
314314+ ];
315315+ in
316316+ _intersection
317317+ (elemAt filesets 0)
318318+ (elemAt filesets 1);
278319279320 /*
280321 Incrementally evaluate and trace a file set in a pretty way.
+110-5
lib/fileset/internal.nix
···461461 else
462462 nonEmpty;
463463464464+ # Transforms the filesetTree of a file set to a shorter base path, e.g.
465465+ # _shortenTreeBase [ "foo" ] (_create /foo/bar null)
466466+ # => { bar = null; }
467467+ _shortenTreeBase = targetBaseComponents: fileset:
468468+ let
469469+ recurse = index:
470470+ # If we haven't reached the required depth yet
471471+ if index < length fileset._internalBaseComponents then
472472+ # Create an attribute set and recurse as the value, this can be lazily evaluated this way
473473+ { ${elemAt fileset._internalBaseComponents index} = recurse (index + 1); }
474474+ else
475475+ # Otherwise we reached the appropriate depth, here's the original tree
476476+ fileset._internalTree;
477477+ in
478478+ recurse (length targetBaseComponents);
479479+480480+ # Transforms the filesetTree of a file set to a longer base path, e.g.
481481+ # _lengthenTreeBase [ "foo" "bar" ] (_create /foo { bar.baz = "regular"; })
482482+ # => { baz = "regular"; }
483483+ _lengthenTreeBase = targetBaseComponents: fileset:
484484+ let
485485+ recurse = index: tree:
486486+ # If the filesetTree is an attribute set and we haven't reached the required depth yet
487487+ if isAttrs tree && index < length targetBaseComponents then
488488+ # Recurse with the tree under the right component (which might not exist)
489489+ recurse (index + 1) (tree.${elemAt targetBaseComponents index} or null)
490490+ else
491491+ # For all values here we can just return the tree itself:
492492+ # tree == null -> the result is also null, everything is excluded
493493+ # tree == "directory" -> the result is also "directory",
494494+ # because the base path is always a directory and everything is included
495495+ # isAttrs tree -> the result is `tree`
496496+ # because we don't need to recurse any more since `index == length longestBaseComponents`
497497+ tree;
498498+ in
499499+ recurse (length fileset._internalBaseComponents) fileset._internalTree;
500500+464501 # Computes the union of a list of filesets.
465502 # The filesets must already be coerced and validated to be in the same filesystem root
466503 # Type: [ Fileset ] -> Fileset
···497534 # So the tree under `/foo/bar` gets nested under `{ bar = ...; ... }`,
498535 # while the tree under `/foo/baz` gets nested under `{ baz = ...; ... }`
499536 # Therefore allowing combined operations over them.
500500- trees = map (fileset:
501501- setAttrByPath
502502- (drop (length commonBaseComponents) fileset._internalBaseComponents)
503503- fileset._internalTree
504504- ) filesetsWithBase;
537537+ trees = map (_shortenTreeBase commonBaseComponents) filesetsWithBase;
505538506539 # Folds all trees together into a single one using _unionTree
507540 # We do not use a fold here because it would cause a thunk build-up
···533566 # The non-null elements have to be attribute sets representing partial trees
534567 # We need to recurse into those
535568 zipAttrsWith (name: _unionTrees) withoutNull;
569569+570570+ # Computes the intersection of a list of filesets.
571571+ # The filesets must already be coerced and validated to be in the same filesystem root
572572+ # Type: Fileset -> Fileset -> Fileset
573573+ _intersection = fileset1: fileset2:
574574+ let
575575+ # The common base components prefix, e.g.
576576+ # (/foo/bar, /foo/bar/baz) -> /foo/bar
577577+ # (/foo/bar, /foo/baz) -> /foo
578578+ commonBaseComponentsLength =
579579+ # TODO: Have a `lib.lists.commonPrefixLength` function such that we don't need the list allocation from commonPrefix here
580580+ length (
581581+ commonPrefix
582582+ fileset1._internalBaseComponents
583583+ fileset2._internalBaseComponents
584584+ );
585585+586586+ # To be able to intersect filesetTree's together, they need to have the same base path.
587587+ # Base paths can be intersected by taking the longest one (if any)
588588+589589+ # The fileset with the longest base, if any, e.g.
590590+ # (/foo/bar, /foo/bar/baz) -> /foo/bar/baz
591591+ # (/foo/bar, /foo/baz) -> null
592592+ longestBaseFileset =
593593+ if commonBaseComponentsLength == length fileset1._internalBaseComponents then
594594+ # The common prefix is the same as the first path, so the second path is equal or longer
595595+ fileset2
596596+ else if commonBaseComponentsLength == length fileset2._internalBaseComponents then
597597+ # The common prefix is the same as the second path, so the first path is longer
598598+ fileset1
599599+ else
600600+ # The common prefix is neither the first nor the second path
601601+ # This means there's no overlap between the two sets
602602+ null;
603603+604604+ # Whether the result should be the empty value without a base
605605+ resultIsEmptyWithoutBase =
606606+ # If either fileset is the empty fileset without a base, the intersection is too
607607+ fileset1._internalIsEmptyWithoutBase
608608+ || fileset2._internalIsEmptyWithoutBase
609609+ # If there is no overlap between the base paths
610610+ || longestBaseFileset == null;
611611+612612+ # Lengthen each fileset's tree to the longest base prefix
613613+ tree1 = _lengthenTreeBase longestBaseFileset._internalBaseComponents fileset1;
614614+ tree2 = _lengthenTreeBase longestBaseFileset._internalBaseComponents fileset2;
615615+616616+ # With two filesetTree's with the same base, we can compute their intersection
617617+ resultTree = _intersectTree tree1 tree2;
618618+ in
619619+ if resultIsEmptyWithoutBase then
620620+ _emptyWithoutBase
621621+ else
622622+ _create longestBaseFileset._internalBase resultTree;
623623+624624+ # The intersection of two filesetTree's with the same base path
625625+ # The second element is only evaluated as much as necessary.
626626+ # Type: filesetTree -> filesetTree -> filesetTree
627627+ _intersectTree = lhs: rhs:
628628+ if isAttrs lhs && isAttrs rhs then
629629+ # Both sides are attribute sets, we can recurse for the attributes existing on both sides
630630+ mapAttrs
631631+ (name: _intersectTree lhs.${name})
632632+ (builtins.intersectAttrs lhs rhs)
633633+ else if lhs == null || isString rhs then
634634+ # If the lhs is null, the result should also be null
635635+ # And if the rhs is the identity element
636636+ # (a string, aka it includes everything), then it's also the lhs
637637+ lhs
638638+ else
639639+ # In all other cases it's the rhs
640640+ rhs;
536641}
+95
lib/fileset/tests.sh
···587587# So, just using 1000 files for now.
588588checkFileset 'unions (mapAttrsToList (name: _: ./. + "/${name}/a") (builtins.readDir ./.))'
589589590590+591591+## lib.fileset.intersection
592592+593593+594594+# Different filesystem roots in root and fileset are not supported
595595+mkdir -p {foo,bar}/mock-root
596596+expectFailure 'with ((import <nixpkgs/lib>).extend (import <nixpkgs/lib/fileset/mock-splitRoot.nix>)).fileset;
597597+ toSource { root = ./.; fileset = intersection ./foo/mock-root ./bar/mock-root; }
598598+' 'lib.fileset.intersection: Filesystem roots are not the same:
599599+\s*first argument: root "'"$work"'/foo/mock-root"
600600+\s*second argument: root "'"$work"'/bar/mock-root"
601601+\s*Different roots are not supported.'
602602+rm -rf -- *
603603+604604+# Coercion errors show the correct context
605605+expectFailure 'toSource { root = ./.; fileset = intersection ./a ./.; }' 'lib.fileset.intersection: first argument \('"$work"'/a\) does not exist.'
606606+expectFailure 'toSource { root = ./.; fileset = intersection ./. ./b; }' 'lib.fileset.intersection: second argument \('"$work"'/b\) does not exist.'
607607+608608+# The tree of later arguments should not be evaluated if a former argument already excludes all files
609609+tree=(
610610+ [a]=0
611611+)
612612+checkFileset 'intersection _emptyWithoutBase (_create ./. (abort "This should not be used!"))'
613613+# We don't have any combinators that can explicitly remove files yet, so we need to rely on internal functions to test this for now
614614+checkFileset 'intersection (_create ./. { a = null; }) (_create ./. { a = abort "This should not be used!"; })'
615615+616616+# If either side is empty, the result is empty
617617+tree=(
618618+ [a]=0
619619+)
620620+checkFileset 'intersection _emptyWithoutBase _emptyWithoutBase'
621621+checkFileset 'intersection _emptyWithoutBase (_create ./. null)'
622622+checkFileset 'intersection (_create ./. null) _emptyWithoutBase'
623623+checkFileset 'intersection (_create ./. null) (_create ./. null)'
624624+625625+# If the intersection base paths are not overlapping, the result is empty and has no base path
626626+mkdir a b c
627627+touch {a,b,c}/x
628628+expectEqual 'toSource { root = ./c; fileset = intersection ./a ./b; }' 'toSource { root = ./c; fileset = _emptyWithoutBase; }'
629629+rm -rf -- *
630630+631631+# If the intersection exists, the resulting base path is the longest of them
632632+mkdir a
633633+touch x a/b
634634+expectEqual 'toSource { root = ./a; fileset = intersection ./a ./.; }' 'toSource { root = ./a; fileset = ./a; }'
635635+expectEqual 'toSource { root = ./a; fileset = intersection ./. ./a; }' 'toSource { root = ./a; fileset = ./a; }'
636636+rm -rf -- *
637637+638638+# Also finds the intersection with null'd filesetTree's
639639+tree=(
640640+ [a]=0
641641+ [b]=1
642642+ [c]=0
643643+)
644644+checkFileset 'intersection (_create ./. { a = "regular"; b = "regular"; c = null; }) (_create ./. { a = null; b = "regular"; c = "regular"; })'
645645+646646+# Actually computes the intersection between files
647647+tree=(
648648+ [a]=0
649649+ [b]=0
650650+ [c]=1
651651+ [d]=1
652652+ [e]=0
653653+ [f]=0
654654+)
655655+checkFileset 'intersection (unions [ ./a ./b ./c ./d ]) (unions [ ./c ./d ./e ./f ])'
656656+657657+tree=(
658658+ [a/x]=0
659659+ [a/y]=0
660660+ [b/x]=1
661661+ [b/y]=1
662662+ [c/x]=0
663663+ [c/y]=0
664664+)
665665+checkFileset 'intersection ./b ./.'
666666+checkFileset 'intersection ./b (unions [ ./a/x ./a/y ./b/x ./b/y ./c/x ./c/y ])'
667667+668668+# Complicated case
669669+tree=(
670670+ [a/x]=0
671671+ [a/b/i]=1
672672+ [c/d/x]=0
673673+ [c/d/f]=1
674674+ [c/x]=0
675675+ [c/e/i]=1
676676+ [c/e/j]=1
677677+)
678678+checkFileset 'intersection (unions [ ./a/b ./c/d ./c/e ]) (unions [ ./a ./c/d/f ./c/e ])'
679679+680680+590681## Tracing
591682592683# The second trace argument is returned
···609700# The empty file set without a base also prints as empty
610701expectTrace '_emptyWithoutBase' '(empty)'
611702expectTrace 'unions [ ]' '(empty)'
703703+mkdir foo bar
704704+touch {foo,bar}/x
705705+expectTrace 'intersection ./foo ./bar' '(empty)'
706706+rm -rf -- *
612707613708# If a directory is fully included, print it as such
614709touch a
···438438 blasProvider = blas.provider;
439439 # To help debug when a package is broken due to CUDA support
440440 inherit brokenConditions;
441441- } // lib.optionalAttrs cudaSupport {
442442- # NOTE: supportedCudaCapabilities isn't computed unless cudaSupport is true, so we can't use
443443- # it in the passthru set above because a downstream package might try to access it even
444444- # when cudaSupport is false. Better to have it missing than null or an empty list by default.
445445- cudaCapabilities = supportedCudaCapabilities;
441441+ cudaCapabilities = if cudaSupport then supportedCudaCapabilities else [ ];
446442 };
447443448444 meta = with lib; {