···96969797## Updating the package set {#updating-the-package-set}
98989999+There is a script and associated environment for regenerating the package
100100+sets and synchronising the rPackages tree to the current CRAN and matching
101101+BIOC release. These scripts are found in the `pkgs/development/r-modules`
102102+directory and executed as follows:
103103+99104```bash
100105nix-shell generate-shell.nix
101106···112117mv bioc-experiment-packages.nix.new bioc-experiment-packages.nix
113118```
114119115115-`generate-r-packages.R <repo>` reads `<repo>-packages.nix`, therefor the renaming.
120120+`generate-r-packages.R <repo>` reads `<repo>-packages.nix`, therefore
121121+the renaming.
116122117117-## Testing if the Nix-expression could be evaluated {#testing-if-the-nix-expression-could-be-evaluated}
118118-119119-```bash
120120-nix-build test-evaluation.nix --dry-run
121121-```
122122-123123-If this exits fine, the expression is ok. If not, you have to edit `default.nix`
123123+Some packages require overrides to specify external dependencies or other
124124+patches and special requirements. These overrides are specified in the
125125+`pkgs/development/r-modules/default.nix` file. As the `*-packages.nix`
126126+contents are automatically generated it should not be edited and broken
127127+builds should be addressed using overrides.
+1-1
nixos/modules/services/mail/postfix.nix
···544544 type = types.lines;
545545 default = "";
546546 description = "
547547- Entries for the virtual alias map, cf. man-page virtual(8).
547547+ Entries for the virtual alias map, cf. man-page virtual(5).
548548 ";
549549 };
550550
···84848585in
8686stdenv.mkDerivation ({
8787- name = "${baseName}-${version}"
8888- + optionalString javacSupport "-javac"
8989- + optionalString odbcSupport "-odbc";
8787+ # name is used instead of pname to
8888+ # - not have to pass pnames as argument
8989+ # - have a separate pname for erlang (main module)
9090+ name = "${baseName}"
9191+ + optionalString javacSupport "_javac"
9292+ + optionalString odbcSupport "_odbc"
9393+ + "-${version}";
90949195 inherit src version;
9296
+41-10
pkgs/development/misc/resholve/README.md
···2233resholve converts bare executable references in shell scripts to absolute
44paths. This will hopefully make its way into the Nixpkgs manual soon, but
55-until then I'll outline how to use the `resholvePackage` function.
55+until then I'll outline how to use the `resholvePackage`, `resholveScript`,
66+and `resholveScriptBin` functions.
6778> Fair warning: resholve does *not* aspire to resolving all valid Shell
89> scripts. It depends on the OSH/Oil parser, which aims to support most (but
···2122> - Packages with scripts that require conflicting directives can use multiple
2223> solutions to resolve the scripts separately, but produce a single package.
23242424-## Basic Example
2525+The `resholveScript` and `resholveScriptBin` functions support a _single_
2626+`solution` attrset. This is basically the same as any single solution in `resholvePackage`, except that it doesn't need a `scripts` attr (it is automatically added).
2727+2828+## Basic `resholvePackage` Example
25292630Here's a simple example from one of my own projects, with annotations:
2731<!--
···6872}
6973```
70747575+## Basic `resholveScript` and `resholveScriptBin` examples
7676+7777+Both of these functions have the same basic API. This example is a little
7878+trivial for now. If you have a real usage that you find helpful, please PR it.
7979+8080+```nix
8181+resholvedScript = resholveScript "name" {
8282+ inputs = [ file ];
8383+ interpreter = "${bash}/bin/bash";
8484+ } ''
8585+ echo "Hello"
8686+ file .
8787+ '';
8888+resholvedScriptBin = resholveScriptBin "name" {
8989+ inputs = [ file ];
9090+ interpreter = "${bash}/bin/bash";
9191+ } ''
9292+ echo "Hello"
9393+ file .
9494+ '';
9595+```
9696+7197## Options
72987399`resholvePackage` maps Nix types/idioms into the flags and environment variables
···79105| inputs | list | packages to resolve executables from |
80106| interpreter | string | 'none' or abspath for shebang |
81107| prologue | file | text to insert before the first code-line |
8282-| epilogue | file | text to isnert after the last code-line |
108108+| epilogue | file | text to insert after the last code-line |
83109| flags | list | strings to pass as flags |
84110| fake | attrset | [directives](#controlling-resolution-with-directives) |
85111| fix | attrset | [directives](#controlling-resolution-with-directives) |
···135161```nix
136162# --fake 'f:setUp;tearDown builtin:setopt source:/etc/bashrc'
137163fake = {
138138- # fake accepts the initial of valid identifier types as a CLI convienience.
164164+ # fake accepts the initial of valid identifier types as a CLI convenience.
139165 # Use full names in the Nix API.
140166 function = [ "setUp" "tearDown" ];
141167 builtin = [ "setopt" ];
142168 source = [ "/etc/bashrc" ];
143169};
144170145145-# --fix 'aliases xargs:ls $GIT:gix'
171171+# --fix 'aliases $GIT:gix /bin/bash'
146172fix = {
147173 # all single-word directives use `true` as value
148174 aliases = true;
149149- xargs = [ "ls" ];
150175 "$GIT" = [ "gix" ];
176176+ "/bin/bash";
151177};
152178153153-# --keep 'which:git;ls .:$HOME $LS:exa /etc/bashrc ~/.bashrc'
179179+# --keep 'source:$HOME /etc/bashrc ~/.bashrc'
154180keep = {
155155- which = [ "git" "ls" ];
156156- "." = [ "$HOME" ];
157157- "$LS" = [ "exa" ];
181181+ source = [ "$HOME" ];
158182 "/etc/bashrc" = true;
159183 "~/.bashrc" = true;
160184};
161185```
186186+187187+> **Note:** For now, at least, you'll need to reference the manpage to completely understand these examples.
162188163189## Controlling nested resolution with lore
164190···176202 either built-in rules for finding the executable, or human triage.
177203- "wrapper" lore maps shell exec wrappers to the programs they exec so
178204 that resholve can substitute an executable's verdict for its wrapper's.
205205+206206+> **Caution:** At least when it comes to common utilities, it's best to treat
207207+> overrides as a stopgap until they can be properly handled in resholve and/or
208208+> binlore. Please report things you have to override and, if possible, help
209209+> get them sorted.
179210180211There will be more mechanisms for controlling this process in the future
181212(and your reports/experiences will play a role in shaping them...) For now,
···4646 nativeBuildInputs = [ git ];
4747 };
48484949+ /*
5050+ Upstream isn't interested in packaging this as a library
5151+ (or accepting all of the patches we need to do so).
5252+ This creates one without disturbing upstream too much.
5353+ */
4954 oildev = python27Packages.buildPythonPackage rec {
5055 pname = "oildev-unstable";
5156 version = "2021-07-14";
···6166 It's not critical to drop most of these; the primary target is
6267 the vendored fork of Python-2.7.13, which is ~ 55M and over 3200
6368 files, dozens of which get interpreter script patches in fixup.
6969+7070+ Note: -f is necessary to keep it from being a pain to update
7171+ hash on rev updates. Command will fail w/o and not print hash.
6472 */
6573 extraPostFetch = ''
6674 rm -rf Python-2.7.13 benchmarks metrics py-yajl rfc gold web testdata services demo devtools cpp
6775 '';
6876 };
6969-7070- # TODO: not sure why I'm having to set this for nix-build...
7171- # can anyone tell if I'm doing something wrong?
7272- SOURCE_DATE_EPOCH = 315532800;
73777478 # patch to support a python package, pass tests on macOS, etc.
7579 patchSrc = fetchFromGitHub {
7680 owner = "abathur";
7781 repo = "nix-py-dev-oil";
7878- rev = "v0.8.12";
7979- hash = "sha256-/EvwxL201lGsioL0lIhzM8VTghe6FuVbc3PBJgY8c8E=";
8282+ rev = "v0.8.12.1";
8383+ hash = "sha256-7JVnosdcvmVFN3h6SIeeqcJFcyFkai//fFuzi7ThNMY=";
8084 };
8185 patches = [
8286 "${patchSrc}/0001-add_setup_py.patch"
···102106 patchShebangs asdl build core doctools frontend native oil_lang
103107 '';
104108105105- # TODO: this may be obsolete?
109109+ /*
110110+ We did convince oil to upstream an env for specifying
111111+ this to support a shell.nix. Would need a patch if they
112112+ later drop this support. See:
113113+ https://github.com/oilshell/oil/blob/46900310c7e4a07a6223eb6c08e4f26460aad285/doctools/cmark.py#L30-L34
114114+ */
106115 _NIX_SHELL_LIBCMARK = "${cmark}/lib/libcmark${stdenv.hostPlatform.extensions.sharedLibrary}";
107116108117 # See earlier note on glibcLocales TODO: verify needed?
···11-{ stdenv, lib, resholve, binlore }:
11+{ stdenv, lib, resholve, resholve-utils }:
2233{ pname
44, src
···99}@attrs:
1010let
1111 inherit stdenv;
1212- /* These functions break up the work of partially validating the
1313- 'solutions' attrset and massaging it into env/cli args.
1414-1515- Note: some of the left-most args do not *have* to be passed as
1616- deep as they are, but I've done so to provide more error context
1717- */
1818-1919- # for brevity / line length
2020- spaces = l: builtins.concatStringsSep " " l;
2121- semicolons = l: builtins.concatStringsSep ";" l;
2222-2323- /* Throw a fit with dotted attr path context */
2424- nope = path: msg:
2525- throw "${builtins.concatStringsSep "." path}: ${msg}";
2626-2727- /* Special-case directive value representations by type */
2828- makeDirective = solution: env: name: val:
2929- if builtins.isInt val then builtins.toString val
3030- else if builtins.isString val then name
3131- else if true == val then name
3232- else if false == val then "" # omit!
3333- else if null == val then "" # omit!
3434- else if builtins.isList val then "${name}:${semicolons val}"
3535- else nope [ solution env name ] "unexpected type: ${builtins.typeOf val}";
3636-3737- /* Build fake/fix/keep directives from Nix types */
3838- makeDirectives = solution: env: val:
3939- lib.mapAttrsToList (makeDirective solution env) val;
4040-4141- /* Special-case value representation by type/name */
4242- makeEnvVal = solution: env: val:
4343- if env == "inputs" then lib.makeBinPath val
4444- else if builtins.isString val then val
4545- else if builtins.isList val then spaces val
4646- else if builtins.isAttrs val then spaces (makeDirectives solution env val)
4747- else nope [ solution env ] "unexpected type: ${builtins.typeOf val}";
4848-4949- /* Shell-format each env value */
5050- shellEnv = solution: env: value:
5151- lib.escapeShellArg (makeEnvVal solution env value);
5252-5353- /* Build a single ENV=val pair */
5454- makeEnv = solution: env: value:
5555- "RESHOLVE_${lib.toUpper env}=${shellEnv solution env value}";
5656-5757- /* Discard attrs claimed by makeArgs */
5858- removeCliArgs = value:
5959- removeAttrs value [ "scripts" "flags" ];
6060-6161- /* Verify required arguments are present */
6262- validateSolution = { scripts, inputs, interpreter, ... }: true;
6363-6464- /* Pull out specific solution keys to build ENV=val pairs */
6565- makeEnvs = solution: value:
6666- spaces (lib.mapAttrsToList (makeEnv solution) (removeCliArgs value));
6767-6868- /* Pull out specific solution keys to build CLI argstring */
6969- makeArgs = { flags ? [ ], scripts, ... }:
7070- spaces (flags ++ scripts);
7171-7272- /* Build a single resholve invocation */
7373- makeInvocation = solution: value:
7474- if validateSolution value then
7575- # we pass resholve a directory
7676- "RESHOLVE_LORE=${binlore.collect { drvs = value.inputs; } } ${makeEnvs solution value} resholve --overwrite ${makeArgs value}"
7777- else throw "invalid solution"; # shouldn't trigger for now
7878-7979- /* Build resholve invocation for each solution. */
8080- makeCommands = solutions:
8181- lib.mapAttrsToList makeInvocation solutions;
82128313 self = (stdenv.mkDerivation ((removeAttrs attrs [ "solutions" ])
8414 // {
8515 inherit pname version src;
8686- buildInputs = [ resholve ];
1616+ buildInputs = (lib.optionals (builtins.hasAttr "buildInputs" attrs) attrs.buildInputs) ++ [ resholve ];
87178818 # enable below for verbose debug info if needed
8919 # supports default python.logging levels
···9929 PS4=$'\x1f'"\033[33m[resholve context]\033[0m "
10030 set -x
10131 : changing directory to $PWD
102102- ${builtins.concatStringsSep "\n" (makeCommands solutions)}
3232+ ${builtins.concatStringsSep "\n" (resholve-utils.makeCommands solutions)}
10333 )
10434 '';
10535 }));
+74
pkgs/development/misc/resholve/resholve-utils.nix
···11+{ lib, resholve, binlore }:
22+33+rec {
44+ /* These functions break up the work of partially validating the
55+ 'solutions' attrset and massaging it into env/cli args.
66+77+ Note: some of the left-most args do not *have* to be passed as
88+ deep as they are, but I've done so to provide more error context
99+ */
1010+1111+ # for brevity / line length
1212+ spaces = l: builtins.concatStringsSep " " l;
1313+ semicolons = l: builtins.concatStringsSep ";" l;
1414+1515+ /* Throw a fit with dotted attr path context */
1616+ nope = path: msg:
1717+ throw "${builtins.concatStringsSep "." path}: ${msg}";
1818+1919+ /* Special-case directive value representations by type */
2020+ makeDirective = solution: env: name: val:
2121+ if builtins.isInt val then builtins.toString val
2222+ else if builtins.isString val then name
2323+ else if true == val then name
2424+ else if false == val then "" # omit!
2525+ else if null == val then "" # omit!
2626+ else if builtins.isList val then "${name}:${semicolons val}"
2727+ else nope [ solution env name ] "unexpected type: ${builtins.typeOf val}";
2828+2929+ /* Build fake/fix/keep directives from Nix types */
3030+ makeDirectives = solution: env: val:
3131+ lib.mapAttrsToList (makeDirective solution env) val;
3232+3333+ /* Special-case value representation by type/name */
3434+ makeEnvVal = solution: env: val:
3535+ if env == "inputs" then lib.makeBinPath val
3636+ else if builtins.isString val then val
3737+ else if builtins.isList val then spaces val
3838+ else if builtins.isAttrs val then spaces (makeDirectives solution env val)
3939+ else nope [ solution env ] "unexpected type: ${builtins.typeOf val}";
4040+4141+ /* Shell-format each env value */
4242+ shellEnv = solution: env: value:
4343+ lib.escapeShellArg (makeEnvVal solution env value);
4444+4545+ /* Build a single ENV=val pair */
4646+ makeEnv = solution: env: value:
4747+ "RESHOLVE_${lib.toUpper env}=${shellEnv solution env value}";
4848+4949+ /* Discard attrs claimed by makeArgs */
5050+ removeCliArgs = value:
5151+ removeAttrs value [ "scripts" "flags" ];
5252+5353+ /* Verify required arguments are present */
5454+ validateSolution = { scripts, inputs, interpreter, ... }: true;
5555+5656+ /* Pull out specific solution keys to build ENV=val pairs */
5757+ makeEnvs = solution: value:
5858+ spaces (lib.mapAttrsToList (makeEnv solution) (removeCliArgs value));
5959+6060+ /* Pull out specific solution keys to build CLI argstring */
6161+ makeArgs = { flags ? [ ], scripts, ... }:
6262+ spaces (flags ++ scripts);
6363+6464+ /* Build a single resholve invocation */
6565+ makeInvocation = solution: value:
6666+ if validateSolution value then
6767+ # we pass resholve a directory
6868+ "RESHOLVE_LORE=${binlore.collect { drvs = value.inputs; } } ${makeEnvs solution value} ${resholve}/bin/resholve --overwrite ${makeArgs value}"
6969+ else throw "invalid solution"; # shouldn't trigger for now
7070+7171+ /* Build resholve invocation for each solution. */
7272+ makeCommands = solutions:
7373+ lib.mapAttrsToList makeInvocation solutions;
7474+}