···11+# Running Tests interactively {#sec-running-nixos-tests-interactively}
22+33+The test itself can be run interactively. This is particularly useful
44+when developing or debugging a test:
55+66+```ShellSession
77+$ nix-build nixos/tests/login.nix -A driverInteractive
88+$ ./result/bin/nixos-test-driver
99+starting VDE switch for network 1
1010+>
1111+```
1212+1313+You can then take any Python statement, e.g.
1414+1515+```py
1616+> start_all()
1717+> test_script()
1818+> machine.succeed("touch /tmp/foo")
1919+> print(machine.succeed("pwd")) # Show stdout of command
2020+```
2121+2222+The function `test_script` executes the entire test script and drops you
2323+back into the test driver command line upon its completion. This allows
2424+you to inspect the state of the VMs after the test (e.g. to debug the
2525+test script).
2626+2727+To just start and experiment with the VMs, run:
2828+2929+```ShellSession
3030+$ nix-build nixos/tests/login.nix -A driverInteractive
3131+$ ./result/bin/nixos-run-vms
3232+```
3333+3434+The script `nixos-run-vms` starts the virtual machines defined by test.
3535+3636+You can re-use the VM states coming from a previous run by setting the
3737+`--keep-vm-state` flag.
3838+3939+```ShellSession
4040+$ ./result/bin/nixos-run-vms --keep-vm-state
4141+```
4242+4343+The machine state is stored in the `$TMPDIR/vm-state-machinename`
4444+directory.
···11-<section xmlns="http://docbook.org/ns/docbook"
22- xmlns:xlink="http://www.w3.org/1999/xlink"
33- xmlns:xi="http://www.w3.org/2001/XInclude"
44- version="5.0"
55- xml:id="sec-running-nixos-tests-interactively">
66- <title>Running Tests interactively</title>
77-88- <para>
99- The test itself can be run interactively. This is particularly useful when
1010- developing or debugging a test:
1111-<screen>
1212-<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driverInteractive
1313-<prompt>$ </prompt>./result/bin/nixos-test-driver
1414-starting VDE switch for network 1
1515-<prompt>></prompt>
1616-</screen>
1717- You can then take any Python statement, e.g.
1818-<screen>
1919-<prompt>></prompt> start_all()
2020-<prompt>></prompt> test_script()
2121-<prompt>></prompt> machine.succeed("touch /tmp/foo")
2222-<prompt>></prompt> print(machine.succeed("pwd")) # Show stdout of command
2323-</screen>
2424- The function <command>test_script</command> executes the entire test script
2525- and drops you back into the test driver command line upon its completion.
2626- This allows you to inspect the state of the VMs after the test (e.g. to debug
2727- the test script).
2828- </para>
2929-3030- <para>
3131- To just start and experiment with the VMs, run:
3232-<screen>
3333-<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driverInteractive
3434-<prompt>$ </prompt>./result/bin/nixos-run-vms
3535-</screen>
3636- The script <command>nixos-run-vms</command> starts the virtual machines
3737- defined by test.
3838- </para>
3939-4040- <para>
4141- You can re-use the VM states coming from a previous run
4242- by setting the <command>--keep-vm-state</command> flag.
4343-<screen>
4444-<prompt>$ </prompt>./result/bin/nixos-run-vms --keep-vm-state
4545-</screen>
4646- The machine state is stored in the
4747- <filename>$TMPDIR/vm-state-</filename><varname>machinename</varname> directory.
4848- </para>
4949-</section>
···11+# Running Tests {#sec-running-nixos-tests}
22+33+You can run tests using `nix-build`. For example, to run the test
44+[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix),
55+you just do:
66+77+```ShellSession
88+$ nix-build '<nixpkgs/nixos/tests/login.nix>'
99+```
1010+1111+or, if you don't want to rely on `NIX_PATH`:
1212+1313+```ShellSession
1414+$ cd /my/nixpkgs/nixos/tests
1515+$ nix-build login.nix
1616+…
1717+running the VM test script
1818+machine: QEMU running (pid 8841)
1919+…
2020+6 out of 6 tests succeeded
2121+```
2222+2323+After building/downloading all required dependencies, this will perform
2424+a build that starts a QEMU/KVM virtual machine containing a NixOS
2525+system. The virtual machine mounts the Nix store of the host; this makes
2626+VM creation very fast, as no disk image needs to be created. Afterwards,
2727+you can view a pretty-printed log of the test:
2828+2929+```ShellSession
3030+$ firefox result/log.html
3131+```
···11-<section xmlns="http://docbook.org/ns/docbook"
22- xmlns:xlink="http://www.w3.org/1999/xlink"
33- xmlns:xi="http://www.w3.org/2001/XInclude"
44- version="5.0"
55- xml:id="sec-running-nixos-tests">
66- <title>Running Tests</title>
77-88- <para>
99- You can run tests using <command>nix-build</command>. For example, to run the
1010- test
1111- <filename
1212-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename>,
1313- you just do:
1414-<screen>
1515-<prompt>$ </prompt>nix-build '<nixpkgs/nixos/tests/login.nix>'
1616-</screen>
1717- or, if you don’t want to rely on <envar>NIX_PATH</envar>:
1818-<screen>
1919-<prompt>$ </prompt>cd /my/nixpkgs/nixos/tests
2020-<prompt>$ </prompt>nix-build login.nix
2121-…
2222-running the VM test script
2323-machine: QEMU running (pid 8841)
2424-…
2525-6 out of 6 tests succeeded
2626-</screen>
2727- After building/downloading all required dependencies, this will perform a
2828- build that starts a QEMU/KVM virtual machine containing a NixOS system. The
2929- virtual machine mounts the Nix store of the host; this makes VM creation very
3030- fast, as no disk image needs to be created. Afterwards, you can view a
3131- pretty-printed log of the test:
3232-<screen>
3333-<prompt>$ </prompt>firefox result/log.html
3434-</screen>
3535- </para>
3636-</section>
···11+# Writing Tests {#sec-writing-nixos-tests}
22+33+A NixOS test is a Nix expression that has the following structure:
44+55+```nix
66+import ./make-test-python.nix {
77+88+ # Either the configuration of a single machine:
99+ machine =
1010+ { config, pkgs, ... }:
1111+ { configuration…
1212+ };
1313+1414+ # Or a set of machines:
1515+ nodes =
1616+ { machine1 =
1717+ { config, pkgs, ... }: { … };
1818+ machine2 =
1919+ { config, pkgs, ... }: { … };
2020+ …
2121+ };
2222+2323+ testScript =
2424+ ''
2525+ Python code…
2626+ '';
2727+}
2828+```
2929+3030+The attribute `testScript` is a bit of Python code that executes the
3131+test (described below). During the test, it will start one or more
3232+virtual machines, the configuration of which is described by the
3333+attribute `machine` (if you need only one machine in your test) or by
3434+the attribute `nodes` (if you need multiple machines). For instance,
3535+[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix)
3636+only needs a single machine to test whether users can log in
3737+on the virtual console, whether device ownership is correctly maintained
3838+when switching between consoles, and so on. On the other hand,
3939+[`nfs/simple.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix),
4040+which tests NFS client and server functionality in the
4141+Linux kernel (including whether locks are maintained across server
4242+crashes), requires three machines: a server and two clients.
4343+4444+There are a few special NixOS configuration options for test VMs:
4545+4646+`virtualisation.memorySize`
4747+4848+: The memory of the VM in megabytes.
4949+5050+`virtualisation.vlans`
5151+5252+: The virtual networks to which the VM is connected. See
5353+ [`nat.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix)
5454+ for an example.
5555+5656+`virtualisation.writableStore`
5757+5858+: By default, the Nix store in the VM is not writable. If you enable
5959+ this option, a writable union file system is mounted on top of the
6060+ Nix store to make it appear writable. This is necessary for tests
6161+ that run Nix operations that modify the store.
6262+6363+For more options, see the module
6464+[`qemu-vm.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix).
6565+6666+The test script is a sequence of Python statements that perform various
6767+actions, such as starting VMs, executing commands in the VMs, and so on.
6868+Each virtual machine is represented as an object stored in the variable
6969+`name` if this is also the identifier of the machine in the declarative
7070+config. If you didn\'t specify multiple machines using the `nodes`
7171+attribute, it is just `machine`. The following example starts the
7272+machine, waits until it has finished booting, then executes a command
7373+and checks that the output is more-or-less correct:
7474+7575+```py
7676+machine.start()
7777+machine.wait_for_unit("default.target")
7878+if not "Linux" in machine.succeed("uname"):
7979+ raise Exception("Wrong OS")
8080+```
8181+8282+The first line is actually unnecessary; machines are implicitly started
8383+when you first execute an action on them (such as `wait_for_unit` or
8484+`succeed`). If you have multiple machines, you can speed up the test by
8585+starting them in parallel:
8686+8787+```py
8888+start_all()
8989+```
9090+9191+The following methods are available on machine objects:
9292+9393+`start`
9494+9595+: Start the virtual machine. This method is asynchronous --- it does
9696+ not wait for the machine to finish booting.
9797+9898+`shutdown`
9999+100100+: Shut down the machine, waiting for the VM to exit.
101101+102102+`crash`
103103+104104+: Simulate a sudden power failure, by telling the VM to exit
105105+ immediately.
106106+107107+`block`
108108+109109+: Simulate unplugging the Ethernet cable that connects the machine to
110110+ the other machines.
111111+112112+`unblock`
113113+114114+: Undo the effect of `block`.
115115+116116+`screenshot`
117117+118118+: Take a picture of the display of the virtual machine, in PNG format.
119119+ The screenshot is linked from the HTML log.
120120+121121+`get_screen_text_variants`
122122+123123+: Return a list of different interpretations of what is currently
124124+ visible on the machine\'s screen using optical character
125125+ recognition. The number and order of the interpretations is not
126126+ specified and is subject to change, but if no exception is raised at
127127+ least one will be returned.
128128+129129+ ::: {.note}
130130+ This requires passing `enableOCR` to the test attribute set.
131131+ :::
132132+133133+`get_screen_text`
134134+135135+: Return a textual representation of what is currently visible on the
136136+ machine\'s screen using optical character recognition.
137137+138138+ ::: {.note}
139139+ This requires passing `enableOCR` to the test attribute set.
140140+ :::
141141+142142+`send_monitor_command`
143143+144144+: Send a command to the QEMU monitor. This is rarely used, but allows
145145+ doing stuff such as attaching virtual USB disks to a running
146146+ machine.
147147+148148+`send_key`
149149+150150+: Simulate pressing keys on the virtual keyboard, e.g.,
151151+ `send_key("ctrl-alt-delete")`.
152152+153153+`send_chars`
154154+155155+: Simulate typing a sequence of characters on the virtual keyboard,
156156+ e.g., `send_chars("foobar\n")` will type the string `foobar`
157157+ followed by the Enter key.
158158+159159+`execute`
160160+161161+: Execute a shell command, returning a list `(status, stdout)`.
162162+163163+`succeed`
164164+165165+: Execute a shell command, raising an exception if the exit status is
166166+ not zero, otherwise returning the standard output. Commands are run
167167+ with `set -euo pipefail` set:
168168+169169+ - If several commands are separated by `;` and one fails, the
170170+ command as a whole will fail.
171171+172172+ - For pipelines, the last non-zero exit status will be returned
173173+ (if there is one, zero will be returned otherwise).
174174+175175+ - Dereferencing unset variables fail the command.
176176+177177+`fail`
178178+179179+: Like `succeed`, but raising an exception if the command returns a zero
180180+ status.
181181+182182+`wait_until_succeeds`
183183+184184+: Repeat a shell command with 1-second intervals until it succeeds.
185185+186186+`wait_until_fails`
187187+188188+: Repeat a shell command with 1-second intervals until it fails.
189189+190190+`wait_for_unit`
191191+192192+: Wait until the specified systemd unit has reached the "active"
193193+ state.
194194+195195+`wait_for_file`
196196+197197+: Wait until the specified file exists.
198198+199199+`wait_for_open_port`
200200+201201+: Wait until a process is listening on the given TCP port (on
202202+ `localhost`, at least).
203203+204204+`wait_for_closed_port`
205205+206206+: Wait until nobody is listening on the given TCP port.
207207+208208+`wait_for_x`
209209+210210+: Wait until the X11 server is accepting connections.
211211+212212+`wait_for_text`
213213+214214+: Wait until the supplied regular expressions matches the textual
215215+ contents of the screen by using optical character recognition (see
216216+ `get_screen_text` and `get_screen_text_variants`).
217217+218218+ ::: {.note}
219219+ This requires passing `enableOCR` to the test attribute set.
220220+ :::
221221+222222+`wait_for_console_text`
223223+224224+: Wait until the supplied regular expressions match a line of the
225225+ serial console output. This method is useful when OCR is not
226226+ possibile or accurate enough.
227227+228228+`wait_for_window`
229229+230230+: Wait until an X11 window has appeared whose name matches the given
231231+ regular expression, e.g., `wait_for_window("Terminal")`.
232232+233233+`copy_from_host`
234234+235235+: Copies a file from host to machine, e.g.,
236236+ `copy_from_host("myfile", "/etc/my/important/file")`.
237237+238238+ The first argument is the file on the host. The file needs to be
239239+ accessible while building the nix derivation. The second argument is
240240+ the location of the file on the machine.
241241+242242+`systemctl`
243243+244244+: Runs `systemctl` commands with optional support for
245245+ `systemctl --user`
246246+247247+ ```py
248248+ machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager`
249249+ machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager`
250250+ ```
251251+252252+`shell_interact`
253253+254254+: Allows you to directly interact with the guest shell. This should
255255+ only be used during test development, not in production tests.
256256+ Killing the interactive session with `Ctrl-d` or `Ctrl-c` also ends
257257+ the guest session.
258258+259259+To test user units declared by `systemd.user.services` the optional
260260+`user` argument can be used:
261261+262262+```py
263263+machine.start()
264264+machine.wait_for_x()
265265+machine.wait_for_unit("xautolock.service", "x-session-user")
266266+```
267267+268268+This applies to `systemctl`, `get_unit_info`, `wait_for_unit`,
269269+`start_job` and `stop_job`.
270270+271271+For faster dev cycles it\'s also possible to disable the code-linters
272272+(this shouldn\'t be commited though):
273273+274274+```nix
275275+import ./make-test-python.nix {
276276+ skipLint = true;
277277+ machine =
278278+ { config, pkgs, ... }:
279279+ { configuration…
280280+ };
281281+282282+ testScript =
283283+ ''
284284+ Python code…
285285+ '';
286286+}
287287+```
288288+289289+This will produce a Nix warning at evaluation time. To fully disable the
290290+linter, wrap the test script in comment directives to disable the Black
291291+linter directly (again, don\'t commit this within the Nixpkgs
292292+repository):
293293+294294+```nix
295295+ testScript =
296296+ ''
297297+ # fmt: off
298298+ Python code…
299299+ # fmt: on
300300+ '';
301301+```
···11-<section xmlns="http://docbook.org/ns/docbook"
22- xmlns:xlink="http://www.w3.org/1999/xlink"
33- xmlns:xi="http://www.w3.org/2001/XInclude"
44- version="5.0"
55- xml:id="sec-writing-nixos-tests">
66- <title>Writing Tests</title>
77-88- <para>
99- A NixOS test is a Nix expression that has the following structure:
1010-<programlisting>
1111-import ./make-test-python.nix {
1212-1313- # Either the configuration of a single machine:
1414- machine =
1515- { config, pkgs, ... }:
1616- { <replaceable>configuration…</replaceable>
1717- };
1818-1919- # Or a set of machines:
2020- nodes =
2121- { <replaceable>machine1</replaceable> =
2222- { config, pkgs, ... }: { <replaceable>…</replaceable> };
2323- <replaceable>machine2</replaceable> =
2424- { config, pkgs, ... }: { <replaceable>…</replaceable> };
2525- …
2626- };
2727-2828- testScript =
2929- ''
3030- <replaceable>Python code…</replaceable>
3131- '';
3232-}
3333-</programlisting>
3434- The attribute <literal>testScript</literal> is a bit of Python code that
3535- executes the test (described below). During the test, it will start one or
3636- more virtual machines, the configuration of which is described by the
3737- attribute <literal>machine</literal> (if you need only one machine in your
3838- test) or by the attribute <literal>nodes</literal> (if you need multiple
3939- machines). For instance,
4040- <filename
4141-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename>
4242- only needs a single machine to test whether users can log in on the virtual
4343- console, whether device ownership is correctly maintained when switching
4444- between consoles, and so on. On the other hand,
4545- <filename
4646-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix">nfs/simple.nix</filename>,
4747- which tests NFS client and server functionality in the Linux kernel
4848- (including whether locks are maintained across server crashes), requires
4949- three machines: a server and two clients.
5050- </para>
5151-5252- <para>
5353- There are a few special NixOS configuration options for test VMs:
5454-<!-- FIXME: would be nice to generate this automatically. -->
5555- <variablelist>
5656- <varlistentry>
5757- <term>
5858- <option>virtualisation.memorySize</option>
5959- </term>
6060- <listitem>
6161- <para>
6262- The memory of the VM in megabytes.
6363- </para>
6464- </listitem>
6565- </varlistentry>
6666- <varlistentry>
6767- <term>
6868- <option>virtualisation.vlans</option>
6969- </term>
7070- <listitem>
7171- <para>
7272- The virtual networks to which the VM is connected. See
7373- <filename
7474- xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename>
7575- for an example.
7676- </para>
7777- </listitem>
7878- </varlistentry>
7979- <varlistentry>
8080- <term>
8181- <option>virtualisation.writableStore</option>
8282- </term>
8383- <listitem>
8484- <para>
8585- By default, the Nix store in the VM is not writable. If you enable this
8686- option, a writable union file system is mounted on top of the Nix store
8787- to make it appear writable. This is necessary for tests that run Nix
8888- operations that modify the store.
8989- </para>
9090- </listitem>
9191- </varlistentry>
9292- </variablelist>
9393- For more options, see the module
9494- <filename
9595-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>.
9696- </para>
9797-9898- <para>
9999- The test script is a sequence of Python statements that perform various
100100- actions, such as starting VMs, executing commands in the VMs, and so on. Each
101101- virtual machine is represented as an object stored in the variable
102102- <literal><replaceable>name</replaceable></literal> if this is also the
103103- identifier of the machine in the declarative config.
104104- If you didn't specify multiple machines using the <literal>nodes</literal>
105105- attribute, it is just <literal>machine</literal>.
106106- The following example starts the machine, waits until it has finished booting,
107107- then executes a command and checks that the output is more-or-less correct:
108108-<programlisting>
109109-machine.start()
110110-machine.wait_for_unit("default.target")
111111-if not "Linux" in machine.succeed("uname"):
112112- raise Exception("Wrong OS")
113113-</programlisting>
114114- The first line is actually unnecessary; machines are implicitly started when
115115- you first execute an action on them (such as <literal>wait_for_unit</literal>
116116- or <literal>succeed</literal>). If you have multiple machines, you can speed
117117- up the test by starting them in parallel:
118118-<programlisting>
119119-start_all()
120120-</programlisting>
121121- </para>
122122-123123- <para>
124124- The following methods are available on machine objects:
125125- <variablelist>
126126- <varlistentry>
127127- <term>
128128- <methodname>start</methodname>
129129- </term>
130130- <listitem>
131131- <para>
132132- Start the virtual machine. This method is asynchronous — it does not
133133- wait for the machine to finish booting.
134134- </para>
135135- </listitem>
136136- </varlistentry>
137137- <varlistentry>
138138- <term>
139139- <methodname>shutdown</methodname>
140140- </term>
141141- <listitem>
142142- <para>
143143- Shut down the machine, waiting for the VM to exit.
144144- </para>
145145- </listitem>
146146- </varlistentry>
147147- <varlistentry>
148148- <term>
149149- <methodname>crash</methodname>
150150- </term>
151151- <listitem>
152152- <para>
153153- Simulate a sudden power failure, by telling the VM to exit immediately.
154154- </para>
155155- </listitem>
156156- </varlistentry>
157157- <varlistentry>
158158- <term>
159159- <methodname>block</methodname>
160160- </term>
161161- <listitem>
162162- <para>
163163- Simulate unplugging the Ethernet cable that connects the machine to the
164164- other machines.
165165- </para>
166166- </listitem>
167167- </varlistentry>
168168- <varlistentry>
169169- <term>
170170- <methodname>unblock</methodname>
171171- </term>
172172- <listitem>
173173- <para>
174174- Undo the effect of <methodname>block</methodname>.
175175- </para>
176176- </listitem>
177177- </varlistentry>
178178- <varlistentry>
179179- <term>
180180- <methodname>screenshot</methodname>
181181- </term>
182182- <listitem>
183183- <para>
184184- Take a picture of the display of the virtual machine, in PNG format. The
185185- screenshot is linked from the HTML log.
186186- </para>
187187- </listitem>
188188- </varlistentry>
189189- <varlistentry>
190190- <term>
191191- <methodname>get_screen_text_variants</methodname>
192192- </term>
193193- <listitem>
194194- <para>
195195- Return a list of different interpretations of what is currently visible
196196- on the machine's screen using optical character recognition. The number
197197- and order of the interpretations is not specified and is subject to
198198- change, but if no exception is raised at least one will be returned.
199199- </para>
200200- <note>
201201- <para>
202202- This requires passing <option>enableOCR</option> to the test attribute
203203- set.
204204- </para>
205205- </note>
206206- </listitem>
207207- </varlistentry>
208208- <varlistentry>
209209- <term>
210210- <methodname>get_screen_text</methodname>
211211- </term>
212212- <listitem>
213213- <para>
214214- Return a textual representation of what is currently visible on the
215215- machine's screen using optical character recognition.
216216- </para>
217217- <note>
218218- <para>
219219- This requires passing <option>enableOCR</option> to the test attribute
220220- set.
221221- </para>
222222- </note>
223223- </listitem>
224224- </varlistentry>
225225- <varlistentry>
226226- <term>
227227- <methodname>send_monitor_command</methodname>
228228- </term>
229229- <listitem>
230230- <para>
231231- Send a command to the QEMU monitor. This is rarely used, but allows doing
232232- stuff such as attaching virtual USB disks to a running machine.
233233- </para>
234234- </listitem>
235235- </varlistentry>
236236- <varlistentry>
237237- <term>
238238- <methodname>send_key</methodname>
239239- </term>
240240- <listitem>
241241- <para>
242242- Simulate pressing keys on the virtual keyboard, e.g.,
243243- <literal>send_key("ctrl-alt-delete")</literal>.
244244- </para>
245245- </listitem>
246246- </varlistentry>
247247- <varlistentry>
248248- <term>
249249- <methodname>send_chars</methodname>
250250- </term>
251251- <listitem>
252252- <para>
253253- Simulate typing a sequence of characters on the virtual keyboard, e.g.,
254254- <literal>send_chars("foobar\n")</literal> will type the string
255255- <literal>foobar</literal> followed by the Enter key.
256256- </para>
257257- </listitem>
258258- </varlistentry>
259259- <varlistentry>
260260- <term>
261261- <methodname>execute</methodname>
262262- </term>
263263- <listitem>
264264- <para>
265265- Execute a shell command, returning a list
266266- <literal>(<replaceable>status</replaceable>,
267267- <replaceable>stdout</replaceable>)</literal>.
268268- </para>
269269- </listitem>
270270- </varlistentry>
271271- <varlistentry>
272272- <term>
273273- <methodname>succeed</methodname>
274274- </term>
275275- <listitem>
276276- <para>
277277- Execute a shell command, raising an exception if the exit status
278278- is not zero, otherwise returning the standard output. Commands
279279- are run with <literal>set -euo pipefail</literal> set:
280280- <itemizedlist>
281281- <listitem>
282282- <para>
283283- If several commands are separated by <literal>;</literal>
284284- and one fails, the command as a whole will fail.
285285- </para>
286286- </listitem>
287287- <listitem>
288288- <para>
289289- For pipelines, the last non-zero exit status will be
290290- returned (if there is one, zero will be returned
291291- otherwise).
292292- </para>
293293- </listitem>
294294- <listitem>
295295- <para>
296296- Dereferencing unset variables fail the command.
297297- </para>
298298- </listitem>
299299- </itemizedlist>
300300- </para>
301301- </listitem>
302302- </varlistentry>
303303- <varlistentry>
304304- <term>
305305- <methodname>fail</methodname>
306306- </term>
307307- <listitem>
308308- <para>
309309- Like <methodname>succeed</methodname>, but raising an exception if the
310310- command returns a zero status.
311311- </para>
312312- </listitem>
313313- </varlistentry>
314314- <varlistentry>
315315- <term>
316316- <methodname>wait_until_succeeds</methodname>
317317- </term>
318318- <listitem>
319319- <para>
320320- Repeat a shell command with 1-second intervals until it succeeds.
321321- </para>
322322- </listitem>
323323- </varlistentry>
324324- <varlistentry>
325325- <term>
326326- <methodname>wait_until_fails</methodname>
327327- </term>
328328- <listitem>
329329- <para>
330330- Repeat a shell command with 1-second intervals until it fails.
331331- </para>
332332- </listitem>
333333- </varlistentry>
334334- <varlistentry>
335335- <term>
336336- <methodname>wait_for_unit</methodname>
337337- </term>
338338- <listitem>
339339- <para>
340340- Wait until the specified systemd unit has reached the “active” state.
341341- </para>
342342- </listitem>
343343- </varlistentry>
344344- <varlistentry>
345345- <term>
346346- <methodname>wait_for_file</methodname>
347347- </term>
348348- <listitem>
349349- <para>
350350- Wait until the specified file exists.
351351- </para>
352352- </listitem>
353353- </varlistentry>
354354- <varlistentry>
355355- <term>
356356- <methodname>wait_for_open_port</methodname>
357357- </term>
358358- <listitem>
359359- <para>
360360- Wait until a process is listening on the given TCP port (on
361361- <literal>localhost</literal>, at least).
362362- </para>
363363- </listitem>
364364- </varlistentry>
365365- <varlistentry>
366366- <term>
367367- <methodname>wait_for_closed_port</methodname>
368368- </term>
369369- <listitem>
370370- <para>
371371- Wait until nobody is listening on the given TCP port.
372372- </para>
373373- </listitem>
374374- </varlistentry>
375375- <varlistentry>
376376- <term>
377377- <methodname>wait_for_x</methodname>
378378- </term>
379379- <listitem>
380380- <para>
381381- Wait until the X11 server is accepting connections.
382382- </para>
383383- </listitem>
384384- </varlistentry>
385385- <varlistentry>
386386- <term>
387387- <methodname>wait_for_text</methodname>
388388- </term>
389389- <listitem>
390390- <para>
391391- Wait until the supplied regular expressions matches the textual contents
392392- of the screen by using optical character recognition (see
393393- <methodname>get_screen_text</methodname> and
394394- <methodname>get_screen_text_variants</methodname>).
395395- </para>
396396- <note>
397397- <para>
398398- This requires passing <option>enableOCR</option> to the test attribute
399399- set.
400400- </para>
401401- </note>
402402- </listitem>
403403- </varlistentry>
404404- <varlistentry>
405405- <term>
406406- <methodname>wait_for_console_text</methodname>
407407- </term>
408408- <listitem>
409409- <para>
410410- Wait until the supplied regular expressions match a line of the serial
411411- console output. This method is useful when OCR is not possibile or
412412- accurate enough.
413413- </para>
414414- </listitem>
415415- </varlistentry>
416416- <varlistentry>
417417- <term>
418418- <methodname>wait_for_window</methodname>
419419- </term>
420420- <listitem>
421421- <para>
422422- Wait until an X11 window has appeared whose name matches the given
423423- regular expression, e.g., <literal>wait_for_window("Terminal")</literal>.
424424- </para>
425425- </listitem>
426426- </varlistentry>
427427- <varlistentry>
428428- <term>
429429- <methodname>copy_from_host</methodname>
430430- </term>
431431- <listitem>
432432- <para>
433433- Copies a file from host to machine, e.g.,
434434- <literal>copy_from_host("myfile", "/etc/my/important/file")</literal>.
435435- </para>
436436- <para>
437437- The first argument is the file on the host. The file needs to be
438438- accessible while building the nix derivation. The second argument is the
439439- location of the file on the machine.
440440- </para>
441441- </listitem>
442442- </varlistentry>
443443- <varlistentry>
444444- <term>
445445- <methodname>systemctl</methodname>
446446- </term>
447447- <listitem>
448448- <para>
449449- Runs <literal>systemctl</literal> commands with optional support for
450450- <literal>systemctl --user</literal>
451451- </para>
452452- <para>
453453-<programlisting>
454454-machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager`
455455-machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager`
456456-</programlisting>
457457- </para>
458458- </listitem>
459459- </varlistentry>
460460- <varlistentry>
461461- <term>
462462- <methodname>shell_interact</methodname>
463463- </term>
464464- <listitem>
465465- <para>
466466- Allows you to directly interact with the guest shell.
467467- This should only be used during test development, not in production tests.
468468- Killing the interactive session with <literal>Ctrl-d</literal> or <literal>Ctrl-c</literal> also ends the guest session.
469469- </para>
470470- </listitem>
471471- </varlistentry>
472472- </variablelist>
473473- </para>
474474-475475- <para>
476476- To test user units declared by <literal>systemd.user.services</literal> the
477477- optional <literal>user</literal> argument can be used:
478478-<programlisting>
479479-machine.start()
480480-machine.wait_for_x()
481481-machine.wait_for_unit("xautolock.service", "x-session-user")
482482-</programlisting>
483483- This applies to <literal>systemctl</literal>, <literal>get_unit_info</literal>,
484484- <literal>wait_for_unit</literal>, <literal>start_job</literal> and
485485- <literal>stop_job</literal>.
486486- </para>
487487-488488- <para>
489489- For faster dev cycles it's also possible to disable the code-linters (this shouldn't
490490- be commited though):
491491-<programlisting>
492492-import ./make-test-python.nix {
493493- skipLint = true;
494494- machine =
495495- { config, pkgs, ... }:
496496- { <replaceable>configuration…</replaceable>
497497- };
498498-499499- testScript =
500500- ''
501501- <replaceable>Python code…</replaceable>
502502- '';
503503-}
504504-</programlisting>
505505- This will produce a Nix warning at evaluation time. To fully disable the
506506- linter, wrap the test script in comment directives to disable the Black linter
507507- directly (again, don't commit this within the Nixpkgs repository):
508508-<programlisting>
509509- testScript =
510510- ''
511511- # fmt: off
512512- <replaceable>Python code…</replaceable>
513513- # fmt: on
514514- '';
515515-</programlisting>
516516- </para>
517517-</section>
···11+<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-running-nixos-tests-interactively">
22+ <title>Running Tests interactively</title>
33+ <para>
44+ The test itself can be run interactively. This is particularly
55+ useful when developing or debugging a test:
66+ </para>
77+ <programlisting>
88+$ nix-build nixos/tests/login.nix -A driverInteractive
99+$ ./result/bin/nixos-test-driver
1010+starting VDE switch for network 1
1111+>
1212+</programlisting>
1313+ <para>
1414+ You can then take any Python statement, e.g.
1515+ </para>
1616+ <programlisting language="python">
1717+> start_all()
1818+> test_script()
1919+> machine.succeed("touch /tmp/foo")
2020+> print(machine.succeed("pwd")) # Show stdout of command
2121+</programlisting>
2222+ <para>
2323+ The function <literal>test_script</literal> executes the entire test
2424+ script and drops you back into the test driver command line upon its
2525+ completion. This allows you to inspect the state of the VMs after
2626+ the test (e.g. to debug the test script).
2727+ </para>
2828+ <para>
2929+ To just start and experiment with the VMs, run:
3030+ </para>
3131+ <programlisting>
3232+$ nix-build nixos/tests/login.nix -A driverInteractive
3333+$ ./result/bin/nixos-run-vms
3434+</programlisting>
3535+ <para>
3636+ The script <literal>nixos-run-vms</literal> starts the virtual
3737+ machines defined by test.
3838+ </para>
3939+ <para>
4040+ You can re-use the VM states coming from a previous run by setting
4141+ the <literal>--keep-vm-state</literal> flag.
4242+ </para>
4343+ <programlisting>
4444+$ ./result/bin/nixos-run-vms --keep-vm-state
4545+</programlisting>
4646+ <para>
4747+ The machine state is stored in the
4848+ <literal>$TMPDIR/vm-state-machinename</literal> directory.
4949+ </para>
5050+</section>
···11+<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-running-nixos-tests">
22+ <title>Running Tests</title>
33+ <para>
44+ You can run tests using <literal>nix-build</literal>. For example,
55+ to run the test
66+ <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix"><literal>login.nix</literal></link>,
77+ you just do:
88+ </para>
99+ <programlisting>
1010+$ nix-build '<nixpkgs/nixos/tests/login.nix>'
1111+</programlisting>
1212+ <para>
1313+ or, if you don’t want to rely on <literal>NIX_PATH</literal>:
1414+ </para>
1515+ <programlisting>
1616+$ cd /my/nixpkgs/nixos/tests
1717+$ nix-build login.nix
1818+…
1919+running the VM test script
2020+machine: QEMU running (pid 8841)
2121+…
2222+6 out of 6 tests succeeded
2323+</programlisting>
2424+ <para>
2525+ After building/downloading all required dependencies, this will
2626+ perform a build that starts a QEMU/KVM virtual machine containing a
2727+ NixOS system. The virtual machine mounts the Nix store of the host;
2828+ this makes VM creation very fast, as no disk image needs to be
2929+ created. Afterwards, you can view a pretty-printed log of the test:
3030+ </para>
3131+ <programlisting>
3232+$ firefox result/log.html
3333+</programlisting>
3434+</section>
···11+<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-writing-nixos-tests">
22+ <title>Writing Tests</title>
33+ <para>
44+ A NixOS test is a Nix expression that has the following structure:
55+ </para>
66+ <programlisting language="bash">
77+import ./make-test-python.nix {
88+99+ # Either the configuration of a single machine:
1010+ machine =
1111+ { config, pkgs, ... }:
1212+ { configuration…
1313+ };
1414+1515+ # Or a set of machines:
1616+ nodes =
1717+ { machine1 =
1818+ { config, pkgs, ... }: { … };
1919+ machine2 =
2020+ { config, pkgs, ... }: { … };
2121+ …
2222+ };
2323+2424+ testScript =
2525+ ''
2626+ Python code…
2727+ '';
2828+}
2929+</programlisting>
3030+ <para>
3131+ The attribute <literal>testScript</literal> is a bit of Python code
3232+ that executes the test (described below). During the test, it will
3333+ start one or more virtual machines, the configuration of which is
3434+ described by the attribute <literal>machine</literal> (if you need
3535+ only one machine in your test) or by the attribute
3636+ <literal>nodes</literal> (if you need multiple machines). For
3737+ instance,
3838+ <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix"><literal>login.nix</literal></link>
3939+ only needs a single machine to test whether users can log in on the
4040+ virtual console, whether device ownership is correctly maintained
4141+ when switching between consoles, and so on. On the other hand,
4242+ <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix"><literal>nfs/simple.nix</literal></link>,
4343+ which tests NFS client and server functionality in the Linux kernel
4444+ (including whether locks are maintained across server crashes),
4545+ requires three machines: a server and two clients.
4646+ </para>
4747+ <para>
4848+ There are a few special NixOS configuration options for test VMs:
4949+ </para>
5050+ <variablelist>
5151+ <varlistentry>
5252+ <term>
5353+ <literal>virtualisation.memorySize</literal>
5454+ </term>
5555+ <listitem>
5656+ <para>
5757+ The memory of the VM in megabytes.
5858+ </para>
5959+ </listitem>
6060+ </varlistentry>
6161+ <varlistentry>
6262+ <term>
6363+ <literal>virtualisation.vlans</literal>
6464+ </term>
6565+ <listitem>
6666+ <para>
6767+ The virtual networks to which the VM is connected. See
6868+ <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix"><literal>nat.nix</literal></link>
6969+ for an example.
7070+ </para>
7171+ </listitem>
7272+ </varlistentry>
7373+ <varlistentry>
7474+ <term>
7575+ <literal>virtualisation.writableStore</literal>
7676+ </term>
7777+ <listitem>
7878+ <para>
7979+ By default, the Nix store in the VM is not writable. If you
8080+ enable this option, a writable union file system is mounted on
8181+ top of the Nix store to make it appear writable. This is
8282+ necessary for tests that run Nix operations that modify the
8383+ store.
8484+ </para>
8585+ </listitem>
8686+ </varlistentry>
8787+ </variablelist>
8888+ <para>
8989+ For more options, see the module
9090+ <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix"><literal>qemu-vm.nix</literal></link>.
9191+ </para>
9292+ <para>
9393+ The test script is a sequence of Python statements that perform
9494+ various actions, such as starting VMs, executing commands in the
9595+ VMs, and so on. Each virtual machine is represented as an object
9696+ stored in the variable <literal>name</literal> if this is also the
9797+ identifier of the machine in the declarative config. If you didn't
9898+ specify multiple machines using the <literal>nodes</literal>
9999+ attribute, it is just <literal>machine</literal>. The following
100100+ example starts the machine, waits until it has finished booting,
101101+ then executes a command and checks that the output is more-or-less
102102+ correct:
103103+ </para>
104104+ <programlisting language="python">
105105+machine.start()
106106+machine.wait_for_unit("default.target")
107107+if not "Linux" in machine.succeed("uname"):
108108+ raise Exception("Wrong OS")
109109+</programlisting>
110110+ <para>
111111+ The first line is actually unnecessary; machines are implicitly
112112+ started when you first execute an action on them (such as
113113+ <literal>wait_for_unit</literal> or <literal>succeed</literal>). If
114114+ you have multiple machines, you can speed up the test by starting
115115+ them in parallel:
116116+ </para>
117117+ <programlisting language="python">
118118+start_all()
119119+</programlisting>
120120+ <para>
121121+ The following methods are available on machine objects:
122122+ </para>
123123+ <variablelist>
124124+ <varlistentry>
125125+ <term>
126126+ <literal>start</literal>
127127+ </term>
128128+ <listitem>
129129+ <para>
130130+ Start the virtual machine. This method is asynchronous — it
131131+ does not wait for the machine to finish booting.
132132+ </para>
133133+ </listitem>
134134+ </varlistentry>
135135+ <varlistentry>
136136+ <term>
137137+ <literal>shutdown</literal>
138138+ </term>
139139+ <listitem>
140140+ <para>
141141+ Shut down the machine, waiting for the VM to exit.
142142+ </para>
143143+ </listitem>
144144+ </varlistentry>
145145+ <varlistentry>
146146+ <term>
147147+ <literal>crash</literal>
148148+ </term>
149149+ <listitem>
150150+ <para>
151151+ Simulate a sudden power failure, by telling the VM to exit
152152+ immediately.
153153+ </para>
154154+ </listitem>
155155+ </varlistentry>
156156+ <varlistentry>
157157+ <term>
158158+ <literal>block</literal>
159159+ </term>
160160+ <listitem>
161161+ <para>
162162+ Simulate unplugging the Ethernet cable that connects the
163163+ machine to the other machines.
164164+ </para>
165165+ </listitem>
166166+ </varlistentry>
167167+ <varlistentry>
168168+ <term>
169169+ <literal>unblock</literal>
170170+ </term>
171171+ <listitem>
172172+ <para>
173173+ Undo the effect of <literal>block</literal>.
174174+ </para>
175175+ </listitem>
176176+ </varlistentry>
177177+ <varlistentry>
178178+ <term>
179179+ <literal>screenshot</literal>
180180+ </term>
181181+ <listitem>
182182+ <para>
183183+ Take a picture of the display of the virtual machine, in PNG
184184+ format. The screenshot is linked from the HTML log.
185185+ </para>
186186+ </listitem>
187187+ </varlistentry>
188188+ <varlistentry>
189189+ <term>
190190+ <literal>get_screen_text_variants</literal>
191191+ </term>
192192+ <listitem>
193193+ <para>
194194+ Return a list of different interpretations of what is
195195+ currently visible on the machine's screen using optical
196196+ character recognition. The number and order of the
197197+ interpretations is not specified and is subject to change, but
198198+ if no exception is raised at least one will be returned.
199199+ </para>
200200+ <note>
201201+ <para>
202202+ This requires passing <literal>enableOCR</literal> to the
203203+ test attribute set.
204204+ </para>
205205+ </note>
206206+ </listitem>
207207+ </varlistentry>
208208+ <varlistentry>
209209+ <term>
210210+ <literal>get_screen_text</literal>
211211+ </term>
212212+ <listitem>
213213+ <para>
214214+ Return a textual representation of what is currently visible
215215+ on the machine's screen using optical character recognition.
216216+ </para>
217217+ <note>
218218+ <para>
219219+ This requires passing <literal>enableOCR</literal> to the
220220+ test attribute set.
221221+ </para>
222222+ </note>
223223+ </listitem>
224224+ </varlistentry>
225225+ <varlistentry>
226226+ <term>
227227+ <literal>send_monitor_command</literal>
228228+ </term>
229229+ <listitem>
230230+ <para>
231231+ Send a command to the QEMU monitor. This is rarely used, but
232232+ allows doing stuff such as attaching virtual USB disks to a
233233+ running machine.
234234+ </para>
235235+ </listitem>
236236+ </varlistentry>
237237+ <varlistentry>
238238+ <term>
239239+ <literal>send_key</literal>
240240+ </term>
241241+ <listitem>
242242+ <para>
243243+ Simulate pressing keys on the virtual keyboard, e.g.,
244244+ <literal>send_key("ctrl-alt-delete")</literal>.
245245+ </para>
246246+ </listitem>
247247+ </varlistentry>
248248+ <varlistentry>
249249+ <term>
250250+ <literal>send_chars</literal>
251251+ </term>
252252+ <listitem>
253253+ <para>
254254+ Simulate typing a sequence of characters on the virtual
255255+ keyboard, e.g.,
256256+ <literal>send_chars("foobar\n")</literal> will type
257257+ the string <literal>foobar</literal> followed by the Enter
258258+ key.
259259+ </para>
260260+ </listitem>
261261+ </varlistentry>
262262+ <varlistentry>
263263+ <term>
264264+ <literal>execute</literal>
265265+ </term>
266266+ <listitem>
267267+ <para>
268268+ Execute a shell command, returning a list
269269+ <literal>(status, stdout)</literal>.
270270+ </para>
271271+ </listitem>
272272+ </varlistentry>
273273+ <varlistentry>
274274+ <term>
275275+ <literal>succeed</literal>
276276+ </term>
277277+ <listitem>
278278+ <para>
279279+ Execute a shell command, raising an exception if the exit
280280+ status is not zero, otherwise returning the standard output.
281281+ Commands are run with <literal>set -euo pipefail</literal>
282282+ set:
283283+ </para>
284284+ <itemizedlist>
285285+ <listitem>
286286+ <para>
287287+ If several commands are separated by <literal>;</literal>
288288+ and one fails, the command as a whole will fail.
289289+ </para>
290290+ </listitem>
291291+ <listitem>
292292+ <para>
293293+ For pipelines, the last non-zero exit status will be
294294+ returned (if there is one, zero will be returned
295295+ otherwise).
296296+ </para>
297297+ </listitem>
298298+ <listitem>
299299+ <para>
300300+ Dereferencing unset variables fail the command.
301301+ </para>
302302+ </listitem>
303303+ </itemizedlist>
304304+ </listitem>
305305+ </varlistentry>
306306+ <varlistentry>
307307+ <term>
308308+ <literal>fail</literal>
309309+ </term>
310310+ <listitem>
311311+ <para>
312312+ Like <literal>succeed</literal>, but raising an exception if
313313+ the command returns a zero status.
314314+ </para>
315315+ </listitem>
316316+ </varlistentry>
317317+ <varlistentry>
318318+ <term>
319319+ <literal>wait_until_succeeds</literal>
320320+ </term>
321321+ <listitem>
322322+ <para>
323323+ Repeat a shell command with 1-second intervals until it
324324+ succeeds.
325325+ </para>
326326+ </listitem>
327327+ </varlistentry>
328328+ <varlistentry>
329329+ <term>
330330+ <literal>wait_until_fails</literal>
331331+ </term>
332332+ <listitem>
333333+ <para>
334334+ Repeat a shell command with 1-second intervals until it fails.
335335+ </para>
336336+ </listitem>
337337+ </varlistentry>
338338+ <varlistentry>
339339+ <term>
340340+ <literal>wait_for_unit</literal>
341341+ </term>
342342+ <listitem>
343343+ <para>
344344+ Wait until the specified systemd unit has reached the
345345+ <quote>active</quote> state.
346346+ </para>
347347+ </listitem>
348348+ </varlistentry>
349349+ <varlistentry>
350350+ <term>
351351+ <literal>wait_for_file</literal>
352352+ </term>
353353+ <listitem>
354354+ <para>
355355+ Wait until the specified file exists.
356356+ </para>
357357+ </listitem>
358358+ </varlistentry>
359359+ <varlistentry>
360360+ <term>
361361+ <literal>wait_for_open_port</literal>
362362+ </term>
363363+ <listitem>
364364+ <para>
365365+ Wait until a process is listening on the given TCP port (on
366366+ <literal>localhost</literal>, at least).
367367+ </para>
368368+ </listitem>
369369+ </varlistentry>
370370+ <varlistentry>
371371+ <term>
372372+ <literal>wait_for_closed_port</literal>
373373+ </term>
374374+ <listitem>
375375+ <para>
376376+ Wait until nobody is listening on the given TCP port.
377377+ </para>
378378+ </listitem>
379379+ </varlistentry>
380380+ <varlistentry>
381381+ <term>
382382+ <literal>wait_for_x</literal>
383383+ </term>
384384+ <listitem>
385385+ <para>
386386+ Wait until the X11 server is accepting connections.
387387+ </para>
388388+ </listitem>
389389+ </varlistentry>
390390+ <varlistentry>
391391+ <term>
392392+ <literal>wait_for_text</literal>
393393+ </term>
394394+ <listitem>
395395+ <para>
396396+ Wait until the supplied regular expressions matches the
397397+ textual contents of the screen by using optical character
398398+ recognition (see <literal>get_screen_text</literal> and
399399+ <literal>get_screen_text_variants</literal>).
400400+ </para>
401401+ <note>
402402+ <para>
403403+ This requires passing <literal>enableOCR</literal> to the
404404+ test attribute set.
405405+ </para>
406406+ </note>
407407+ </listitem>
408408+ </varlistentry>
409409+ <varlistentry>
410410+ <term>
411411+ <literal>wait_for_console_text</literal>
412412+ </term>
413413+ <listitem>
414414+ <para>
415415+ Wait until the supplied regular expressions match a line of
416416+ the serial console output. This method is useful when OCR is
417417+ not possibile or accurate enough.
418418+ </para>
419419+ </listitem>
420420+ </varlistentry>
421421+ <varlistentry>
422422+ <term>
423423+ <literal>wait_for_window</literal>
424424+ </term>
425425+ <listitem>
426426+ <para>
427427+ Wait until an X11 window has appeared whose name matches the
428428+ given regular expression, e.g.,
429429+ <literal>wait_for_window("Terminal")</literal>.
430430+ </para>
431431+ </listitem>
432432+ </varlistentry>
433433+ <varlistentry>
434434+ <term>
435435+ <literal>copy_from_host</literal>
436436+ </term>
437437+ <listitem>
438438+ <para>
439439+ Copies a file from host to machine, e.g.,
440440+ <literal>copy_from_host("myfile", "/etc/my/important/file")</literal>.
441441+ </para>
442442+ <para>
443443+ The first argument is the file on the host. The file needs to
444444+ be accessible while building the nix derivation. The second
445445+ argument is the location of the file on the machine.
446446+ </para>
447447+ </listitem>
448448+ </varlistentry>
449449+ <varlistentry>
450450+ <term>
451451+ <literal>systemctl</literal>
452452+ </term>
453453+ <listitem>
454454+ <para>
455455+ Runs <literal>systemctl</literal> commands with optional
456456+ support for <literal>systemctl --user</literal>
457457+ </para>
458458+ <programlisting language="python">
459459+machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager`
460460+machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager`
461461+</programlisting>
462462+ </listitem>
463463+ </varlistentry>
464464+ <varlistentry>
465465+ <term>
466466+ <literal>shell_interact</literal>
467467+ </term>
468468+ <listitem>
469469+ <para>
470470+ Allows you to directly interact with the guest shell. This
471471+ should only be used during test development, not in production
472472+ tests. Killing the interactive session with
473473+ <literal>Ctrl-d</literal> or <literal>Ctrl-c</literal> also
474474+ ends the guest session.
475475+ </para>
476476+ </listitem>
477477+ </varlistentry>
478478+ </variablelist>
479479+ <para>
480480+ To test user units declared by
481481+ <literal>systemd.user.services</literal> the optional
482482+ <literal>user</literal> argument can be used:
483483+ </para>
484484+ <programlisting language="python">
485485+machine.start()
486486+machine.wait_for_x()
487487+machine.wait_for_unit("xautolock.service", "x-session-user")
488488+</programlisting>
489489+ <para>
490490+ This applies to <literal>systemctl</literal>,
491491+ <literal>get_unit_info</literal>, <literal>wait_for_unit</literal>,
492492+ <literal>start_job</literal> and <literal>stop_job</literal>.
493493+ </para>
494494+ <para>
495495+ For faster dev cycles it's also possible to disable the code-linters
496496+ (this shouldn't be commited though):
497497+ </para>
498498+ <programlisting language="bash">
499499+import ./make-test-python.nix {
500500+ skipLint = true;
501501+ machine =
502502+ { config, pkgs, ... }:
503503+ { configuration…
504504+ };
505505+506506+ testScript =
507507+ ''
508508+ Python code…
509509+ '';
510510+}
511511+</programlisting>
512512+ <para>
513513+ This will produce a Nix warning at evaluation time. To fully disable
514514+ the linter, wrap the test script in comment directives to disable
515515+ the Black linter directly (again, don't commit this within the
516516+ Nixpkgs repository):
517517+ </para>
518518+ <programlisting language="bash">
519519+ testScript =
520520+ ''
521521+ # fmt: off
522522+ Python code…
523523+ # fmt: on
524524+ '';
525525+</programlisting>
526526+</section>
···2233stdenv.mkDerivation rec {
44 pname = "lean";
55- version = "3.30.0";
55+ version = "3.31.0";
6677 src = fetchFromGitHub {
88 owner = "leanprover-community";
···1111 # from. this is then used to check whether an olean file should be
1212 # rebuilt. don't use a tag as rev because this will get replaced into
1313 # src/githash.h.in in preConfigure.
1414- rev = "a5822ea47ebc52eec6323d8f1b60f6ec025daf99";
1515- sha256 = "sha256-gJhbkl19iilNyfCt2TfPmghYA3yCjg6kS+yk/x/k14Y=";
1414+ rev = "333783350cd3fe38f25fed1da7d6a433d8f85b77";
1515+ sha256 = "sha256-N8Ju7pSGssvt84/0e1o6G/p7fWM1c0Mzw+ftL1/++J4=";
1616 };
17171818 nativeBuildInputs = [ cmake ];
···2626}:
27272828let
2929- version = "1.8.0";
2929+ version = "1.8.1";
30303131 # build stimuli file for PGO build and the script to generate it
3232 # independently of the foot's build, so we can cache the result
···94949595 src = fetchzip {
9696 url = "https://codeberg.org/dnkl/${pname}/archive/${version}.tar.gz";
9797- sha256 = "07irlhkvziv51cp5zn1yz8ljfnrnfjcykv5pgfwmpslw3nl5szxv";
9797+ sha256 = "0yrz7n0wls8g8w7ja934icwxmng3sxh70x87qmzc9c9cb1wyd989";
9898 };
9999100100 nativeBuildInputs = [
+4-4
pkgs/data/misc/hackage/pin.json
···11{
22- "commit": "8005ce7c7ba90fa92db65f86c544623353a96cf8",
33- "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/8005ce7c7ba90fa92db65f86c544623353a96cf8.tar.gz",
44- "sha256": "1kzhh7h0csb0vh9avbjsm6hziaa3lbpmzp4pkij4s3bbl4l664aa",
55- "msg": "Update from Hackage at 2021-06-22T07:13:30Z"
22+ "commit": "1567e96c400fcd62dfc0d9412881591d6e1e9f22",
33+ "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/1567e96c400fcd62dfc0d9412881591d6e1e9f22.tar.gz",
44+ "sha256": "04z26ypfp3nip2x6gwrv5k1lmckmmi03ry3z97syc72qqj59n9hq",
55+ "msg": "Update from Hackage at 2021-06-26T12:56:56Z"
66}
···173173 hls-brittany-plugin = dontCheck super.hls-brittany-plugin;
174174 hls-fourmolu-plugin = dontCheck super.hls-fourmolu-plugin;
175175 hls-module-name-plugin = dontCheck super.hls-module-name-plugin;
176176+ hls-splice-plugin = dontCheck super.hls-splice-plugin;
176177177178 # We are lacking pure pgrep at the moment for tests to work
178179 tmp-postgres = dontCheck super.tmp-postgres;