tjh.dev
1<chapter xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="chap-functions">
4
5<title>Functions reference</title>
6
7<para>
8 The nixpkgs repository has several utility functions to manipulate Nix expressions.
9</para>
10
11<section xml:id="sec-pkgs-overridePackages">
12 <title>pkgs.overridePackages</title>
13
14 <para>
15 This function inside the nixpkgs expression (<varname>pkgs</varname>)
16 can be used to override the set of packages itself.
17 </para>
18 <para>
19 Warning: this function is expensive and must not be used from within
20 the nixpkgs repository.
21 </para>
22 <para>
23 Example usage:
24
25 <programlisting>let
26 pkgs = import <nixpkgs> {};
27 newpkgs = pkgs.overridePackages (self: super: {
28 foo = super.foo.override { ... };
29 };
30in ...</programlisting>
31 </para>
32
33 <para>
34 The resulting <varname>newpkgs</varname> will have the new <varname>foo</varname>
35 expression, and all other expressions depending on <varname>foo</varname> will also
36 use the new <varname>foo</varname> expression.
37 </para>
38
39 <para>
40 The behavior of this function is similar to <link
41 linkend="sec-modify-via-packageOverrides">config.packageOverrides</link>.
42 </para>
43
44 <para>
45 The <varname>self</varname> parameter refers to the final package set with the
46 applied overrides. Using this parameter may lead to infinite recursion if not
47 used consciously.
48 </para>
49
50 <para>
51 The <varname>super</varname> parameter refers to the old package set.
52 It's equivalent to <varname>pkgs</varname> in the above example.
53 </para>
54
55</section>
56
57<section xml:id="sec-pkg-override">
58 <title><pkg>.override</title>
59
60 <para>
61 The function <varname>override</varname> is usually available for all the
62 derivations in the nixpkgs expression (<varname>pkgs</varname>).
63 </para>
64 <para>
65 It is used to override the arguments passed to a function.
66 </para>
67 <para>
68 Example usages:
69
70 <programlisting>pkgs.foo.override { arg1 = val1; arg2 = val2; ... }</programlisting>
71 <programlisting>pkgs.overridePackages (self: super: {
72 foo = super.foo.override { barSupport = true ; };
73})</programlisting>
74 <programlisting>mypkg = pkgs.callPackage ./mypkg.nix {
75 mydep = pkgs.mydep.override { ... };
76})</programlisting>
77 </para>
78
79 <para>
80 In the first example, <varname>pkgs.foo</varname> is the result of a function call
81 with some default arguments, usually a derivation.
82 Using <varname>pkgs.foo.override</varname> will call the same function with
83 the given new arguments.
84 </para>
85
86</section>
87
88<section xml:id="sec-pkg-overrideDerivation">
89 <title><pkg>.overrideDerivation</title>
90
91 <warning>
92 <para>Do not use this function in Nixpkgs. Because it breaks
93 package abstraction and doesn’t provide error checking for
94 function arguments, it is only intended for ad-hoc customisation
95 (such as in <filename>~/.nixpkgs/config.nix</filename>).</para>
96 </warning>
97
98 <para>
99 The function <varname>overrideDerivation</varname> is usually available for all the
100 derivations in the nixpkgs expression (<varname>pkgs</varname>).
101 </para>
102 <para>
103 It is used to create a new derivation by overriding the attributes of
104 the original derivation according to the given function.
105 </para>
106
107 <para>
108 Example usage:
109
110 <programlisting>mySed = pkgs.gnused.overrideDerivation (oldAttrs: {
111 name = "sed-4.2.2-pre";
112 src = fetchurl {
113 url = ftp://alpha.gnu.org/gnu/sed/sed-4.2.2-pre.tar.bz2;
114 sha256 = "11nq06d131y4wmf3drm0yk502d2xc6n5qy82cg88rb9nqd2lj41k";
115 };
116 patches = [];
117});</programlisting>
118 </para>
119
120 <para>
121 In the above example, the name, src and patches of the derivation
122 will be overridden, while all other attributes will be retained from the
123 original derivation.
124 </para>
125
126 <para>
127 The argument <varname>oldAttrs</varname> is used to refer to the attribute set of
128 the original derivation.
129 </para>
130
131</section>
132
133<section xml:id="sec-lib-makeOverridable">
134 <title>lib.makeOverridable</title>
135
136 <para>
137 The function <varname>lib.makeOverridable</varname> is used to make the result
138 of a function easily customizable. This utility only makes sense for functions
139 that accept an argument set and return an attribute set.
140 </para>
141
142 <para>
143 Example usage:
144
145 <programlisting>f = { a, b }: { result = a+b; }
146c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>
147
148 </para>
149
150 <para>
151 The variable <varname>c</varname> is the value of the <varname>f</varname> function
152 applied with some default arguments. Hence the value of <varname>c.result</varname>
153 is <literal>3</literal>, in this example.
154 </para>
155
156 <para>
157 The variable <varname>c</varname> however also has some additional functions, like
158 <link linkend="sec-pkg-override">c.override</link> which can be used to
159 override the default arguments. In this example the value of
160 <varname>(c.override { a = 4; }).result</varname> is 6.
161 </para>
162
163</section>
164
165
166<section xml:id="sec-fhs-environments">
167 <title>buildFHSChrootEnv/buildFHSUserEnv</title>
168
169 <para>
170 <function>buildFHSChrootEnv</function> and
171 <function>buildFHSUserEnv</function> provide a way to build and run
172 FHS-compatible lightweight sandboxes. They get their own isolated root with
173 binded <filename>/nix/store</filename>, so their footprint in terms of disk
174 space needed is quite small. This allows one to run software which is hard or
175 unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions,
176 games distributed as tarballs, software with integrity checking and/or external
177 self-updated binaries.
178 </para>
179
180 <para>
181 <function>buildFHSChrootEnv</function> allows to create persistent
182 environments, which can be constructed, deconstructed and entered by
183 multiple users at once. A downside is that it requires
184 <literal>root</literal> access for both those who create and destroy and
185 those who enter it. It can be useful to create environments for daemons that
186 one can enter and observe.
187 </para>
188
189 <para>
190 <function>buildFHSUserEnv</function> uses Linux namespaces feature to create
191 temporary lightweight environments which are destroyed after all child
192 processes exit. It does not require root access, and can be useful to create
193 sandboxes and wrap applications.
194 </para>
195
196 <para>
197 Those functions both rely on <function>buildFHSEnv</function>, which creates
198 an actual directory structure given a list of necessary packages and extra
199 build commands.
200 <function>buildFHSChrootEnv</function> and <function>buildFHSUserEnv</function>
201 both accept those arguments which are passed to
202 <function>buildFHSEnv</function>:
203 </para>
204
205 <variablelist>
206 <varlistentry>
207 <term><literal>name</literal></term>
208
209 <listitem><para>Environment name.</para></listitem>
210 </varlistentry>
211
212 <varlistentry>
213 <term><literal>targetPkgs</literal></term>
214
215 <listitem><para>Packages to be installed for the main host's architecture
216 (i.e. x86_64 on x86_64 installations).</para></listitem>
217 </varlistentry>
218
219 <varlistentry>
220 <term><literal>multiPkgs</literal></term>
221
222 <listitem><para>Packages to be installed for all architectures supported by
223 a host (i.e. i686 and x86_64 on x86_64 installations).</para></listitem>
224 </varlistentry>
225
226 <varlistentry>
227 <term><literal>extraBuildCommands</literal></term>
228
229 <listitem><para>Additional commands to be executed for finalizing the
230 directory structure.</para></listitem>
231 </varlistentry>
232
233 <varlistentry>
234 <term><literal>extraBuildCommandsMulti</literal></term>
235
236 <listitem><para>Like <literal>extraBuildCommandsMulti</literal>, but
237 executed only on multilib architectures.</para></listitem>
238 </varlistentry>
239 </variablelist>
240
241 <para>
242 Additionally, <function>buildFHSUserEnv</function> accepts
243 <literal>runScript</literal> parameter, which is a command that would be
244 executed inside the sandbox and passed all the command line arguments. It
245 default to <literal>bash</literal>.
246 </para>
247 <para>
248 It also uses <literal>CHROOTENV_EXTRA_BINDS</literal> environment variable
249 for binding extra directories in the sandbox to outside places. The format of
250 the variable is <literal>/mnt=test-mnt:/data</literal>, where
251 <literal>/mnt</literal> would be mounted as <literal>/test-mnt</literal>
252 and <literal>/data</literal> would be mounted as <literal>/data</literal>.
253 <literal>extraBindMounts</literal> array argument to
254 <function>buildFHSUserEnv</function> function is prepended to this variable.
255 Latter entries take priority if defined several times -- i.e. in case of
256 <literal>/data=data1:/data=data2</literal> the actual bind path would be
257 <literal>/data2</literal>.
258 </para>
259 <para>
260 One can create a simple environment using a <literal>shell.nix</literal>
261 like that:
262 </para>
263
264<programlisting><![CDATA[
265{ pkgs ? import <nixpkgs> {} }:
266
267(pkgs.buildFHSUserEnv {
268 name = "simple-x11-env";
269 targetPkgs = pkgs: (with pkgs;
270 [ udev
271 alsaLib
272 ]) ++ (with pkgs.xorg;
273 [ libX11
274 libXcursor
275 libXrandr
276 ]);
277 multiPkgs = pkgs: (with pkgs;
278 [ udev
279 alsaLib
280 ]);
281 runScript = "bash";
282}).env
283]]></programlisting>
284
285 <para>
286 Running <literal>nix-shell</literal> would then drop you into a shell with
287 these libraries and binaries available. You can use this to run
288 closed-source applications which expect FHS structure without hassles:
289 simply change <literal>runScript</literal> to the application path,
290 e.g. <filename>./bin/start.sh</filename> -- relative paths are supported.
291 </para>
292</section>
293
294<section xml:id="sec-pkgs-dockerTools">
295 <title>pkgs.dockerTools</title>
296
297 <para>
298 <varname>pkgs.dockerTools</varname> is a set of functions for creating and
299 manipulating Docker images according to the
300 <link xlink:href="https://github.com/docker/docker/blob/master/image/spec/v1.md#docker-image-specification-v100">
301 Docker Image Specification v1.0.0
302 </link>. Docker itself is not used to perform any of the operations done by these
303 functions.
304 </para>
305
306 <warning>
307 <para>
308 The <varname>dockerTools</varname> API is unstable and may be subject to
309 backwards-incompatible changes in the future.
310 </para>
311 </warning>
312
313 <section xml:id="ssec-pkgs-dockerTools-buildImage">
314 <title>buildImage</title>
315
316 <para>
317 This function is analogous to the <command>docker build</command> command,
318 in that can used to build a Docker-compatible repository tarball containing
319 a single image with one or multiple layers. As such, the result
320 is suitable for being loaded in Docker with <command>docker load</command>.
321 </para>
322
323 <para>
324 The parameters of <varname>buildImage</varname> with relative example values are
325 described below:
326 </para>
327
328 <example xml:id='ex-dockerTools-buildImage'><title>Docker build</title>
329 <programlisting>
330 buildImage {
331 name = "redis"; <co xml:id='ex-dockerTools-buildImage-1' />
332 tag = "latest"; <co xml:id='ex-dockerTools-buildImage-2' />
333
334 fromImage = someBaseImage; <co xml:id='ex-dockerTools-buildImage-3' />
335 fromImageName = null; <co xml:id='ex-dockerTools-buildImage-4' />
336 fromImageTag = "latest"; <co xml:id='ex-dockerTools-buildImage-5' />
337
338 contents = pkgs.redis; <co xml:id='ex-dockerTools-buildImage-6' />
339 runAsRoot = '' <co xml:id='ex-dockerTools-buildImage-runAsRoot' />
340 #!${stdenv.shell}
341 mkdir -p /data
342 '';
343
344 config = { <co xml:id='ex-dockerTools-buildImage-8' />
345 Cmd = [ "/bin/redis-server" ];
346 WorkingDir = "/data";
347 Volumes = {
348 "/data" = {};
349 };
350 };
351 }
352 </programlisting>
353 </example>
354
355 <para>The above example will build a Docker image <literal>redis/latest</literal>
356 from the given base image. Loading and running this image in Docker results in
357 <literal>redis-server</literal> being started automatically.
358 </para>
359
360 <calloutlist>
361 <callout arearefs='ex-dockerTools-buildImage-1'>
362 <para>
363 <varname>name</varname> specifies the name of the resulting image.
364 This is the only required argument for <varname>buildImage</varname>.
365 </para>
366 </callout>
367
368 <callout arearefs='ex-dockerTools-buildImage-2'>
369 <para>
370 <varname>tag</varname> specifies the tag of the resulting image.
371 By default it's <literal>latest</literal>.
372 </para>
373 </callout>
374
375 <callout arearefs='ex-dockerTools-buildImage-3'>
376 <para>
377 <varname>fromImage</varname> is the repository tarball containing the base image.
378 It must be a valid Docker image, such as exported by <command>docker save</command>.
379 By default it's <literal>null</literal>, which can be seen as equivalent
380 to <literal>FROM scratch</literal> of a <filename>Dockerfile</filename>.
381 </para>
382 </callout>
383
384 <callout arearefs='ex-dockerTools-buildImage-4'>
385 <para>
386 <varname>fromImageName</varname> can be used to further specify
387 the base image within the repository, in case it contains multiple images.
388 By default it's <literal>null</literal>, in which case
389 <varname>buildImage</varname> will peek the first image available
390 in the repository.
391 </para>
392 </callout>
393
394 <callout arearefs='ex-dockerTools-buildImage-5'>
395 <para>
396 <varname>fromImageTag</varname> can be used to further specify the tag
397 of the base image within the repository, in case an image contains multiple tags.
398 By default it's <literal>null</literal>, in which case
399 <varname>buildImage</varname> will peek the first tag available for the base image.
400 </para>
401 </callout>
402
403 <callout arearefs='ex-dockerTools-buildImage-6'>
404 <para>
405 <varname>contents</varname> is a derivation that will be copied in the new
406 layer of the resulting image. This can be similarly seen as
407 <command>ADD contents/ /</command> in a <filename>Dockerfile</filename>.
408 By default it's <literal>null</literal>.
409 </para>
410 </callout>
411
412 <callout arearefs='ex-dockerTools-buildImage-runAsRoot'>
413 <para>
414 <varname>runAsRoot</varname> is a bash script that will run as root
415 in an environment that overlays the existing layers of the base image with
416 the new resulting layer, including the previously copied
417 <varname>contents</varname> derivation.
418 This can be similarly seen as
419 <command>RUN ...</command> in a <filename>Dockerfile</filename>.
420
421 <note>
422 <para>
423 Using this parameter requires the <literal>kvm</literal>
424 device to be available.
425 </para>
426 </note>
427 </para>
428 </callout>
429
430 <callout arearefs='ex-dockerTools-buildImage-8'>
431 <para>
432 <varname>config</varname> is used to specify the configuration of the
433 containers that will be started off the built image in Docker.
434 The available options are listed in the
435 <link xlink:href="https://github.com/docker/docker/blob/master/image/spec/v1.md#container-runconfig-field-descriptions">
436 Docker Image Specification v1.0.0
437 </link>.
438 </para>
439 </callout>
440
441 </calloutlist>
442
443 <para>
444 After the new layer has been created, its closure
445 (to which <varname>contents</varname>, <varname>config</varname> and
446 <varname>runAsRoot</varname> contribute) will be copied in the layer itself.
447 Only new dependencies that are not already in the existing layers will be copied.
448 </para>
449
450 <para>
451 At the end of the process, only one new single layer will be produced and
452 added to the resulting image.
453 </para>
454
455 <para>
456 The resulting repository will only list the single image
457 <varname>image/tag</varname>. In the case of <xref linkend='ex-dockerTools-buildImage'/>
458 it would be <varname>redis/latest</varname>.
459 </para>
460
461 <para>
462 It is possible to inspect the arguments with which an image was built
463 using its <varname>buildArgs</varname> attribute.
464 </para>
465
466 </section>
467
468 <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry">
469 <title>pullImage</title>
470
471 <para>
472 This function is analogous to the <command>docker pull</command> command,
473 in that can be used to fetch a Docker image from a Docker registry.
474 Currently only registry <literal>v1</literal> is supported.
475 By default <link xlink:href="https://hub.docker.com/">Docker Hub</link>
476 is used to pull images.
477 </para>
478
479 <para>
480 Its parameters are described in the example below:
481 </para>
482
483 <example xml:id='ex-dockerTools-pullImage'><title>Docker pull</title>
484 <programlisting>
485 pullImage {
486 imageName = "debian"; <co xml:id='ex-dockerTools-pullImage-1' />
487 imageTag = "jessie"; <co xml:id='ex-dockerTools-pullImage-2' />
488 imageId = null; <co xml:id='ex-dockerTools-pullImage-3' />
489 sha256 = "1bhw5hkz6chrnrih0ymjbmn69hyfriza2lr550xyvpdrnbzr4gk2"; <co xml:id='ex-dockerTools-pullImage-4' />
490
491 indexUrl = "https://index.docker.io"; <co xml:id='ex-dockerTools-pullImage-5' />
492 registryVersion = "v1";
493 }
494 </programlisting>
495 </example>
496
497 <calloutlist>
498 <callout arearefs='ex-dockerTools-pullImage-1'>
499 <para>
500 <varname>imageName</varname> specifies the name of the image to be downloaded,
501 which can also include the registry namespace (e.g. <literal>library/debian</literal>).
502 This argument is required.
503 </para>
504 </callout>
505
506 <callout arearefs='ex-dockerTools-pullImage-2'>
507 <para>
508 <varname>imageTag</varname> specifies the tag of the image to be downloaded.
509 By default it's <literal>latest</literal>.
510 </para>
511 </callout>
512
513 <callout arearefs='ex-dockerTools-pullImage-3'>
514 <para>
515 <varname>imageId</varname>, if specified this exact image will be fetched, instead
516 of <varname>imageName/imageTag</varname>. However, the resulting repository
517 will still be named <varname>imageName/imageTag</varname>.
518 By default it's <literal>null</literal>.
519 </para>
520 </callout>
521
522 <callout arearefs='ex-dockerTools-pullImage-4'>
523 <para>
524 <varname>sha256</varname> is the checksum of the whole fetched image.
525 This argument is required.
526 </para>
527
528 <note>
529 <para>The checksum is computed on the unpacked directory, not on the final tarball.</para>
530 </note>
531
532 </callout>
533
534 <callout arearefs='ex-dockerTools-pullImage-5'>
535 <para>
536 In the above example the default values are shown for the variables
537 <varname>indexUrl</varname> and <varname>registryVersion</varname>.
538 Hence by default the Docker.io registry is used to pull the images.
539 </para>
540 </callout>
541 </calloutlist>
542
543 </section>
544
545 <section xml:id="ssec-pkgs-dockerTools-exportImage">
546 <title>exportImage</title>
547
548 <para>
549 This function is analogous to the <command>docker export</command> command,
550 in that can used to flatten a Docker image that contains multiple layers.
551 It is in fact the result of the merge of all the layers of the image.
552 As such, the result is suitable for being imported in Docker
553 with <command>docker import</command>.
554 </para>
555
556 <note>
557 <para>
558 Using this function requires the <literal>kvm</literal>
559 device to be available.
560 </para>
561 </note>
562
563 <para>
564 The parameters of <varname>exportImage</varname> are the following:
565 </para>
566
567 <example xml:id='ex-dockerTools-exportImage'><title>Docker export</title>
568 <programlisting>
569 exportImage {
570 fromImage = someLayeredImage;
571 fromImageName = null;
572 fromImageTag = null;
573
574 name = someLayeredImage.name;
575 }
576 </programlisting>
577 </example>
578
579 <para>
580 The parameters relative to the base image have the same synopsis as
581 described in <xref linkend='ssec-pkgs-dockerTools-buildImage'/>, except that
582 <varname>fromImage</varname> is the only required argument in this case.
583 </para>
584
585 <para>
586 The <varname>name</varname> argument is the name of the derivation output,
587 which defaults to <varname>fromImage.name</varname>.
588 </para>
589 </section>
590
591 <section xml:id="ssec-pkgs-dockerTools-shadowSetup">
592 <title>shadowSetup</title>
593
594 <para>
595 This constant string is a helper for setting up the base files for managing
596 users and groups, only if such files don't exist already.
597 It is suitable for being used in a
598 <varname>runAsRoot</varname> <xref linkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like
599 in the example below:
600 </para>
601
602 <example xml:id='ex-dockerTools-shadowSetup'><title>Shadow base files</title>
603 <programlisting>
604 buildImage {
605 name = "shadow-basic";
606
607 runAsRoot = ''
608 #!${stdenv.shell}
609 ${shadowSetup}
610 groupadd -r redis
611 useradd -r -g redis redis
612 mkdir /data
613 chown redis:redis /data
614 '';
615 }
616 </programlisting>
617 </example>
618
619 <para>
620 Creating base files like <literal>/etc/passwd</literal> or
621 <literal>/etc/login.defs</literal> are necessary for shadow-utils to
622 manipulate users and groups.
623 </para>
624
625 </section>
626
627</section>
628
629</chapter>