tjh.dev
1<section xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xmlns:xi="http://www.w3.org/2001/XInclude"
4 xml:id="sec-pkgs-dockerTools">
5 <title>pkgs.dockerTools</title>
6
7 <para>
8 <varname>pkgs.dockerTools</varname> is a set of functions for creating and
9 manipulating Docker images according to the
10 <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120">
11 Docker Image Specification v1.2.0 </link>. Docker itself is not used to
12 perform any of the operations done by these functions.
13 </para>
14
15 <warning>
16 <para>
17 The <varname>dockerTools</varname> API is unstable and may be subject to
18 backwards-incompatible changes in the future.
19 </para>
20 </warning>
21
22 <section xml:id="ssec-pkgs-dockerTools-buildImage">
23 <title>buildImage</title>
24
25 <para>
26 This function is analogous to the <command>docker build</command> command,
27 in that it can be used to build a Docker-compatible repository tarball containing
28 a single image with one or multiple layers. As such, the result is suitable
29 for being loaded in Docker with <command>docker load</command>.
30 </para>
31
32 <para>
33 The parameters of <varname>buildImage</varname> with relative example values
34 are described below:
35 </para>
36
37 <example xml:id='ex-dockerTools-buildImage'>
38 <title>Docker build</title>
39<programlisting>
40buildImage {
41 name = "redis"; <co xml:id='ex-dockerTools-buildImage-1' />
42 tag = "latest"; <co xml:id='ex-dockerTools-buildImage-2' />
43
44 fromImage = someBaseImage; <co xml:id='ex-dockerTools-buildImage-3' />
45 fromImageName = null; <co xml:id='ex-dockerTools-buildImage-4' />
46 fromImageTag = "latest"; <co xml:id='ex-dockerTools-buildImage-5' />
47
48 contents = pkgs.redis; <co xml:id='ex-dockerTools-buildImage-6' />
49 runAsRoot = '' <co xml:id='ex-dockerTools-buildImage-runAsRoot' />
50 #!${pkgs.runtimeShell}
51 mkdir -p /data
52 '';
53
54 config = { <co xml:id='ex-dockerTools-buildImage-8' />
55 Cmd = [ "/bin/redis-server" ];
56 WorkingDir = "/data";
57 Volumes = {
58 "/data" = {};
59 };
60 };
61}
62</programlisting>
63 </example>
64
65 <para>
66 The above example will build a Docker image <literal>redis/latest</literal>
67 from the given base image. Loading and running this image in Docker results
68 in <literal>redis-server</literal> being started automatically.
69 </para>
70
71 <calloutlist>
72 <callout arearefs='ex-dockerTools-buildImage-1'>
73 <para>
74 <varname>name</varname> specifies the name of the resulting image. This is
75 the only required argument for <varname>buildImage</varname>.
76 </para>
77 </callout>
78 <callout arearefs='ex-dockerTools-buildImage-2'>
79 <para>
80 <varname>tag</varname> specifies the tag of the resulting image. By
81 default it's <literal>null</literal>, which indicates that the nix output
82 hash will be used as tag.
83 </para>
84 </callout>
85 <callout arearefs='ex-dockerTools-buildImage-3'>
86 <para>
87 <varname>fromImage</varname> is the repository tarball containing the base
88 image. It must be a valid Docker image, such as exported by
89 <command>docker save</command>. By default it's <literal>null</literal>,
90 which can be seen as equivalent to <literal>FROM scratch</literal> of a
91 <filename>Dockerfile</filename>.
92 </para>
93 </callout>
94 <callout arearefs='ex-dockerTools-buildImage-4'>
95 <para>
96 <varname>fromImageName</varname> can be used to further specify the base
97 image within the repository, in case it contains multiple images. By
98 default it's <literal>null</literal>, in which case
99 <varname>buildImage</varname> will peek the first image available in the
100 repository.
101 </para>
102 </callout>
103 <callout arearefs='ex-dockerTools-buildImage-5'>
104 <para>
105 <varname>fromImageTag</varname> can be used to further specify the tag of
106 the base image within the repository, in case an image contains multiple
107 tags. By default it's <literal>null</literal>, in which case
108 <varname>buildImage</varname> will peek the first tag available for the
109 base image.
110 </para>
111 </callout>
112 <callout arearefs='ex-dockerTools-buildImage-6'>
113 <para>
114 <varname>contents</varname> is a derivation that will be copied in the new
115 layer of the resulting image. This can be similarly seen as <command>ADD
116 contents/ /</command> in a <filename>Dockerfile</filename>. By default
117 it's <literal>null</literal>.
118 </para>
119 </callout>
120 <callout arearefs='ex-dockerTools-buildImage-runAsRoot'>
121 <para>
122 <varname>runAsRoot</varname> is a bash script that will run as root in an
123 environment that overlays the existing layers of the base image with the
124 new resulting layer, including the previously copied
125 <varname>contents</varname> derivation. This can be similarly seen as
126 <command>RUN ...</command> in a <filename>Dockerfile</filename>.
127 <note>
128 <para>
129 Using this parameter requires the <literal>kvm</literal> device to be
130 available.
131 </para>
132 </note>
133 </para>
134 </callout>
135 <callout arearefs='ex-dockerTools-buildImage-8'>
136 <para>
137 <varname>config</varname> is used to specify the configuration of the
138 containers that will be started off the built image in Docker. The
139 available options are listed in the
140 <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions">
141 Docker Image Specification v1.2.0 </link>.
142 </para>
143 </callout>
144 </calloutlist>
145
146 <para>
147 After the new layer has been created, its closure (to which
148 <varname>contents</varname>, <varname>config</varname> and
149 <varname>runAsRoot</varname> contribute) will be copied in the layer itself.
150 Only new dependencies that are not already in the existing layers will be
151 copied.
152 </para>
153
154 <para>
155 At the end of the process, only one new single layer will be produced and
156 added to the resulting image.
157 </para>
158
159 <para>
160 The resulting repository will only list the single image
161 <varname>image/tag</varname>. In the case of
162 <xref linkend='ex-dockerTools-buildImage'/> it would be
163 <varname>redis/latest</varname>.
164 </para>
165
166 <para>
167 It is possible to inspect the arguments with which an image was built using
168 its <varname>buildArgs</varname> attribute.
169 </para>
170
171 <note>
172 <para>
173 If you see errors similar to <literal>getProtocolByName: does not exist (no
174 such protocol name: tcp)</literal> you may need to add
175 <literal>pkgs.iana-etc</literal> to <varname>contents</varname>.
176 </para>
177 </note>
178
179 <note>
180 <para>
181 If you see errors similar to <literal>Error_Protocol ("certificate has
182 unknown CA",True,UnknownCa)</literal> you may need to add
183 <literal>pkgs.cacert</literal> to <varname>contents</varname>.
184 </para>
185 </note>
186
187 <example xml:id="example-pkgs-dockerTools-buildImage-creation-date">
188 <title>Impurely Defining a Docker Layer's Creation Date</title>
189 <para>
190 By default <function>buildImage</function> will use a static date of one
191 second past the UNIX Epoch. This allows <function>buildImage</function> to
192 produce binary reproducible images. When listing images with
193 <command>docker images</command>, the newly created images will be
194 listed like this:
195 </para>
196<screen><![CDATA[
197$ docker images
198REPOSITORY TAG IMAGE ID CREATED SIZE
199hello latest 08c791c7846e 48 years ago 25.2MB
200]]></screen>
201 <para>
202 You can break binary reproducibility but have a sorted, meaningful
203 <literal>CREATED</literal> column by setting <literal>created</literal> to
204 <literal>now</literal>.
205 </para>
206<programlisting><![CDATA[
207pkgs.dockerTools.buildImage {
208 name = "hello";
209 tag = "latest";
210 created = "now";
211 contents = pkgs.hello;
212
213 config.Cmd = [ "/bin/hello" ];
214}
215]]></programlisting>
216 <para>
217 and now the Docker CLI will display a reasonable date and sort the images
218 as expected:
219<screen><![CDATA[
220$ docker images
221REPOSITORY TAG IMAGE ID CREATED SIZE
222hello latest de2bf4786de6 About a minute ago 25.2MB
223]]></screen>
224 however, the produced images will not be binary reproducible.
225 </para>
226 </example>
227 </section>
228
229 <section xml:id="ssec-pkgs-dockerTools-buildLayeredImage">
230 <title>buildLayeredImage</title>
231
232 <para>
233 Create a Docker image with many of the store paths being on their own layer
234 to improve sharing between images.
235 </para>
236
237 <variablelist>
238 <varlistentry>
239 <term>
240 <varname>name</varname>
241 </term>
242 <listitem>
243 <para>
244 The name of the resulting image.
245 </para>
246 </listitem>
247 </varlistentry>
248 <varlistentry>
249 <term>
250 <varname>tag</varname> <emphasis>optional</emphasis>
251 </term>
252 <listitem>
253 <para>
254 Tag of the generated image.
255 </para>
256 <para>
257 <emphasis>Default:</emphasis> the output path's hash
258 </para>
259 </listitem>
260 </varlistentry>
261 <varlistentry>
262 <term>
263 <varname>contents</varname> <emphasis>optional</emphasis>
264 </term>
265 <listitem>
266 <para>
267 Top level paths in the container. Either a single derivation, or a list
268 of derivations.
269 </para>
270 <para>
271 <emphasis>Default:</emphasis> <literal>[]</literal>
272 </para>
273 </listitem>
274 </varlistentry>
275 <varlistentry>
276 <term>
277 <varname>config</varname> <emphasis>optional</emphasis>
278 </term>
279 <listitem>
280 <para>
281 Run-time configuration of the container. A full list of the options are
282 available at in the
283 <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions">
284 Docker Image Specification v1.2.0 </link>.
285 </para>
286 <para>
287 <emphasis>Default:</emphasis> <literal>{}</literal>
288 </para>
289 </listitem>
290 </varlistentry>
291 <varlistentry>
292 <term>
293 <varname>created</varname> <emphasis>optional</emphasis>
294 </term>
295 <listitem>
296 <para>
297 Date and time the layers were created. Follows the same
298 <literal>now</literal> exception supported by
299 <literal>buildImage</literal>.
300 </para>
301 <para>
302 <emphasis>Default:</emphasis> <literal>1970-01-01T00:00:01Z</literal>
303 </para>
304 </listitem>
305 </varlistentry>
306 <varlistentry>
307 <term>
308 <varname>maxLayers</varname> <emphasis>optional</emphasis>
309 </term>
310 <listitem>
311 <para>
312 Maximum number of layers to create.
313 </para>
314 <para>
315 <emphasis>Default:</emphasis> <literal>24</literal>
316 </para>
317 </listitem>
318 </varlistentry>
319 </variablelist>
320
321 <section xml:id="dockerTools-buildLayeredImage-arg-contents">
322 <title>Behavior of <varname>contents</varname> in the final image</title>
323
324 <para>
325 Each path directly listed in <varname>contents</varname> will have a
326 symlink in the root of the image.
327 </para>
328
329 <para>
330 For example:
331<programlisting><![CDATA[
332pkgs.dockerTools.buildLayeredImage {
333 name = "hello";
334 contents = [ pkgs.hello ];
335}
336]]></programlisting>
337 will create symlinks for all the paths in the <literal>hello</literal>
338 package:
339<screen><![CDATA[
340/bin/hello -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/bin/hello
341/share/info/hello.info -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/share/info/hello.info
342/share/locale/bg/LC_MESSAGES/hello.mo -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/share/locale/bg/LC_MESSAGES/hello.mo
343]]></screen>
344 </para>
345 </section>
346
347 <section xml:id="dockerTools-buildLayeredImage-arg-config">
348 <title>Automatic inclusion of <varname>config</varname> references</title>
349
350 <para>
351 The closure of <varname>config</varname> is automatically included in the
352 closure of the final image.
353 </para>
354
355 <para>
356 This allows you to make very simple Docker images with very little code.
357 This container will start up and run <command>hello</command>:
358<programlisting><![CDATA[
359pkgs.dockerTools.buildLayeredImage {
360 name = "hello";
361 config.Cmd = [ "${pkgs.hello}/bin/hello" ];
362}
363]]></programlisting>
364 </para>
365 </section>
366
367 <section xml:id="dockerTools-buildLayeredImage-arg-maxLayers">
368 <title>Adjusting <varname>maxLayers</varname></title>
369
370 <para>
371 Increasing the <varname>maxLayers</varname> increases the number of layers
372 which have a chance to be shared between different images.
373 </para>
374
375 <para>
376 Modern Docker installations support up to 128 layers, however older
377 versions support as few as 42.
378 </para>
379
380 <para>
381 If the produced image will not be extended by other Docker builds, it is
382 safe to set <varname>maxLayers</varname> to <literal>128</literal>. However
383 it will be impossible to extend the image further.
384 </para>
385
386 <para>
387 The first (<literal>maxLayers-2</literal>) most "popular" paths will have
388 their own individual layers, then layer #<literal>maxLayers-1</literal>
389 will contain all the remaining "unpopular" paths, and finally layer
390 #<literal>maxLayers</literal> will contain the Image configuration.
391 </para>
392
393 <para>
394 Docker's Layers are not inherently ordered, they are content-addressable
395 and are not explicitly layered until they are composed in to an Image.
396 </para>
397 </section>
398 </section>
399
400 <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry">
401 <title>pullImage</title>
402
403 <para>
404 This function is analogous to the <command>docker pull</command> command, in
405 that it can be used to pull a Docker image from a Docker registry. By default
406 <link xlink:href="https://hub.docker.com/">Docker Hub</link> is used to pull
407 images.
408 </para>
409
410 <para>
411 Its parameters are described in the example below:
412 </para>
413
414 <example xml:id='ex-dockerTools-pullImage'>
415 <title>Docker pull</title>
416<programlisting>
417pullImage {
418 imageName = "nixos/nix"; <co xml:id='ex-dockerTools-pullImage-1' />
419 imageDigest = "sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b"; <co xml:id='ex-dockerTools-pullImage-2' />
420 finalImageTag = "1.11"; <co xml:id='ex-dockerTools-pullImage-3' />
421 sha256 = "0mqjy3zq2v6rrhizgb9nvhczl87lcfphq9601wcprdika2jz7qh8"; <co xml:id='ex-dockerTools-pullImage-4' />
422 os = "linux"; <co xml:id='ex-dockerTools-pullImage-5' />
423 arch = "x86_64"; <co xml:id='ex-dockerTools-pullImage-6' />
424}
425</programlisting>
426 </example>
427
428 <calloutlist>
429 <callout arearefs='ex-dockerTools-pullImage-1'>
430 <para>
431 <varname>imageName</varname> specifies the name of the image to be
432 downloaded, which can also include the registry namespace (e.g.
433 <literal>nixos</literal>). This argument is required.
434 </para>
435 </callout>
436 <callout arearefs='ex-dockerTools-pullImage-2'>
437 <para>
438 <varname>imageDigest</varname> specifies the digest of the image to be
439 downloaded. Skopeo can be used to get the digest of an image, with its
440 <varname>inspect</varname> subcommand. Since a given
441 <varname>imageName</varname> may transparently refer to a manifest list of
442 images which support multiple architectures and/or operating systems,
443 supply the `--override-os` and `--override-arch` arguments to specify
444 exactly which image you want. By default it will match the OS and
445 architecture of the host the command is run on.
446<programlisting>
447$ nix-shell --packages skopeo jq --command "skopeo --override-os linux --override-arch x86_64 inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'"
448sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b
449</programlisting>
450 This argument is required.
451 </para>
452 </callout>
453 <callout arearefs='ex-dockerTools-pullImage-3'>
454 <para>
455 <varname>finalImageTag</varname>, if specified, this is the tag of the
456 image to be created. Note it is never used to fetch the image since we
457 prefer to rely on the immutable digest ID. By default it's
458 <literal>latest</literal>.
459 </para>
460 </callout>
461 <callout arearefs='ex-dockerTools-pullImage-4'>
462 <para>
463 <varname>sha256</varname> is the checksum of the whole fetched image. This
464 argument is required.
465 </para>
466 </callout>
467 <callout arearefs='ex-dockerTools-pullImage-5'>
468 <para>
469 <varname>os</varname>, if specified, is the operating system of the
470 fetched image. By default it's <literal>linux</literal>.
471 </para>
472 </callout>
473 <callout arearefs='ex-dockerTools-pullImage-6'>
474 <para>
475 <varname>arch</varname>, if specified, is the cpu architecture of the
476 fetched image. By default it's <literal>x86_64</literal>.
477 </para>
478 </callout>
479 </calloutlist>
480 </section>
481
482 <section xml:id="ssec-pkgs-dockerTools-exportImage">
483 <title>exportImage</title>
484
485 <para>
486 This function is analogous to the <command>docker export</command> command,
487 in that it can be used to flatten a Docker image that contains multiple layers. It
488 is in fact the result of the merge of all the layers of the image. As such,
489 the result is suitable for being imported in Docker with <command>docker
490 import</command>.
491 </para>
492
493 <note>
494 <para>
495 Using this function requires the <literal>kvm</literal> device to be
496 available.
497 </para>
498 </note>
499
500 <para>
501 The parameters of <varname>exportImage</varname> are the following:
502 </para>
503
504 <example xml:id='ex-dockerTools-exportImage'>
505 <title>Docker export</title>
506<programlisting>
507exportImage {
508 fromImage = someLayeredImage;
509 fromImageName = null;
510 fromImageTag = null;
511
512 name = someLayeredImage.name;
513}
514 </programlisting>
515 </example>
516
517 <para>
518 The parameters relative to the base image have the same synopsis as
519 described in <xref linkend='ssec-pkgs-dockerTools-buildImage'/>, except that
520 <varname>fromImage</varname> is the only required argument in this case.
521 </para>
522
523 <para>
524 The <varname>name</varname> argument is the name of the derivation output,
525 which defaults to <varname>fromImage.name</varname>.
526 </para>
527 </section>
528
529 <section xml:id="ssec-pkgs-dockerTools-shadowSetup">
530 <title>shadowSetup</title>
531
532 <para>
533 This constant string is a helper for setting up the base files for managing
534 users and groups, only if such files don't exist already. It is suitable for
535 being used in a <varname>runAsRoot</varname>
536 <xref linkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like
537 in the example below:
538 </para>
539
540 <example xml:id='ex-dockerTools-shadowSetup'>
541 <title>Shadow base files</title>
542<programlisting>
543buildImage {
544 name = "shadow-basic";
545
546 runAsRoot = ''
547 #!${pkgs.runtimeShell}
548 ${shadowSetup}
549 groupadd -r redis
550 useradd -r -g redis redis
551 mkdir /data
552 chown redis:redis /data
553 '';
554}
555</programlisting>
556 </example>
557
558 <para>
559 Creating base files like <literal>/etc/passwd</literal> or
560 <literal>/etc/login.defs</literal> is necessary for shadow-utils to
561 manipulate users and groups.
562 </para>
563 </section>
564</section>