tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Documentation: KUnit: Added KUnit Architecture
Describe the components of KUnit and how the kernel mode parts
interact with kunit_tool.
Signed-off-by: Harinder Singh <sharinder@google.com>
Co-developed-by: Marco Elver <elver@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20211217044911.798817-4-sharinder@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
authored by
Harinder Singh
and committed by
Jonathan Corbet
bc145b37
c48b9ef1
+288
+204
Documentation/dev-tools/kunit/architecture.rst
+204
Documentation/dev-tools/kunit/architecture.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
==================
4
+
KUnit Architecture
5
+
==================
6
+
7
+
The KUnit architecture can be divided into two parts:
8
+
9
+
- Kernel testing library
10
+
- kunit_tool (Command line test harness)
11
+
12
+
In-Kernel Testing Framework
13
+
===========================
14
+
15
+
The kernel testing library supports KUnit tests written in C using
16
+
KUnit. KUnit tests are kernel code. KUnit does several things:
17
+
18
+
- Organizes tests
19
+
- Reports test results
20
+
- Provides test utilities
21
+
22
+
Test Cases
23
+
----------
24
+
25
+
The fundamental unit in KUnit is the test case. The KUnit test cases are
26
+
grouped into KUnit suites. A KUnit test case is a function with type
27
+
signature ``void (*)(struct kunit *test)``.
28
+
These test case functions are wrapped in a struct called
29
+
``struct kunit_case``. For code, see:
30
+
31
+
.. kernel-doc:: include/kunit/test.h
32
+
:identifiers: kunit_case
33
+
34
+
.. note:
35
+
``generate_params`` is optional for non-parameterized tests.
36
+
37
+
Each KUnit test case gets a ``struct kunit`` context
38
+
object passed to it that tracks a running test. The KUnit assertion
39
+
macros and other KUnit utilities use the ``struct kunit`` context
40
+
object. As an exception, there are two fields:
41
+
42
+
- ``->priv``: The setup functions can use it to store arbitrary test
43
+
user data.
44
+
45
+
- ``->param_value``: It contains the parameter value which can be
46
+
retrieved in the parameterized tests.
47
+
48
+
Test Suites
49
+
-----------
50
+
51
+
A KUnit suite includes a collection of test cases. The KUnit suites
52
+
are represented by the ``struct kunit_suite``. For example:
53
+
54
+
.. code-block:: c
55
+
56
+
static struct kunit_case example_test_cases[] = {
57
+
KUNIT_CASE(example_test_foo),
58
+
KUNIT_CASE(example_test_bar),
59
+
KUNIT_CASE(example_test_baz),
60
+
{}
61
+
};
62
+
63
+
static struct kunit_suite example_test_suite = {
64
+
.name = "example",
65
+
.init = example_test_init,
66
+
.exit = example_test_exit,
67
+
.test_cases = example_test_cases,
68
+
};
69
+
kunit_test_suite(example_test_suite);
70
+
71
+
In the above example, the test suite ``example_test_suite``, runs the
72
+
test cases ``example_test_foo``, ``example_test_bar``, and
73
+
``example_test_baz``. Before running the test, the ``example_test_init``
74
+
is called and after running the test, ``example_test_exit`` is called.
75
+
The ``kunit_test_suite(example_test_suite)`` registers the test suite
76
+
with the KUnit test framework.
77
+
78
+
Executor
79
+
--------
80
+
81
+
The KUnit executor can list and run built-in KUnit tests on boot.
82
+
The Test suites are stored in a linker section
83
+
called ``.kunit_test_suites``. For code, see:
84
+
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
85
+
The linker section consists of an array of pointers to
86
+
``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
87
+
macro. To run all tests compiled into the kernel, the KUnit executor
88
+
iterates over the linker section array.
89
+
90
+
.. kernel-figure:: kunit_suitememorydiagram.svg
91
+
:alt: KUnit Suite Memory
92
+
93
+
KUnit Suite Memory Diagram
94
+
95
+
On the kernel boot, the KUnit executor uses the start and end addresses
96
+
of this section to iterate over and run all tests. For code, see:
97
+
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c
98
+
99
+
When built as a module, the ``kunit_test_suites()`` macro defines a
100
+
``module_init()`` function, which runs all the tests in the compilation
101
+
unit instead of utilizing the executor.
102
+
103
+
In KUnit tests, some error classes do not affect other tests
104
+
or parts of the kernel, each KUnit case executes in a separate thread
105
+
context. For code, see:
106
+
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
107
+
108
+
Assertion Macros
109
+
----------------
110
+
111
+
KUnit tests verify state using expectations/assertions.
112
+
All expectations/assertions are formatted as:
113
+
``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, property[, message])``
114
+
115
+
- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
116
+
expectation.
117
+
118
+
- For an expectation, if the check fails, marks the test as failed
119
+
and logs the failure.
120
+
121
+
- An assertion, on failure, causes the test case to terminate
122
+
immediately.
123
+
124
+
- Assertions call function:
125
+
``void __noreturn kunit_abort(struct kunit *)``.
126
+
127
+
- ``kunit_abort`` calls function:
128
+
``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.
129
+
130
+
- ``kunit_try_catch_throw`` calls function:
131
+
``void complete_and_exit(struct completion *, long) __noreturn;``
132
+
and terminates the special thread context.
133
+
134
+
- ``<op>`` denotes a check with options: ``TRUE`` (supplied property
135
+
has the boolean value “true”), ``EQ`` (two supplied properties are
136
+
equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
137
+
contain an “err” value).
138
+
139
+
- ``[_MSG]`` prints a custom message on failure.
140
+
141
+
Test Result Reporting
142
+
---------------------
143
+
KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
144
+
https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
145
+
KTAP (yet to be standardized format) works with KUnit and Kselftest.
146
+
The KUnit executor prints KTAP results to dmesg, and debugfs
147
+
(if configured).
148
+
149
+
Parameterized Tests
150
+
-------------------
151
+
152
+
Each KUnit parameterized test is associated with a collection of
153
+
parameters. The test is invoked multiple times, once for each parameter
154
+
value and the parameter is stored in the ``param_value`` field.
155
+
The test case includes a ``KUNIT_CASE_PARAM()`` macro that accepts a
156
+
generator function.
157
+
The generator function is passed the previous parameter and returns the next
158
+
parameter. It also provides a macro to generate common-case generators based on
159
+
arrays.
160
+
161
+
For code, see:
162
+
163
+
.. kernel-doc:: include/kunit/test.h
164
+
:identifiers: KUNIT_ARRAY_PARAM
165
+
166
+
167
+
kunit_tool (Command Line Test Harness)
168
+
======================================
169
+
170
+
kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
171
+
that can be used to configure, build, exec, parse and run (runs other
172
+
commands in order) test results. You can either run KUnit tests using
173
+
kunit_tool or can include KUnit in kernel and parse manually.
174
+
175
+
- ``configure`` command generates the kernel ``.config`` from a
176
+
``.kunitconfig`` file (and any architecture-specific options).
177
+
For some architectures, additional config options are specified in the
178
+
``qemu_config`` Python script
179
+
(For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
180
+
It parses both the existing ``.config`` and the ``.kunitconfig`` files
181
+
and ensures that ``.config`` is a superset of ``.kunitconfig``.
182
+
If this is not the case, it will combine the two and run
183
+
``make olddefconfig`` to regenerate the ``.config`` file. It then
184
+
verifies that ``.config`` is now a superset. This checks if all
185
+
Kconfig dependencies are correctly specified in ``.kunitconfig``.
186
+
``kunit_config.py`` includes the parsing Kconfigs code. The code which
187
+
runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
188
+
invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
189
+
generate a ``.config`` file.
190
+
- ``build`` runs ``make`` on the kernel tree with required options
191
+
(depends on the architecture and some options, for example: build_dir)
192
+
and reports any errors.
193
+
To build a KUnit kernel from the current ``.config``, you can use the
194
+
``build`` argument: ``./tools/testing/kunit/kunit.py build``.
195
+
- ``exec`` command executes kernel results either directly (using
196
+
User-mode Linux configuration), or via an emulator such
197
+
as QEMU. It reads results from the log via standard
198
+
output (stdout), and passes them to ``parse`` to be parsed.
199
+
If you already have built a kernel with built-in KUnit tests,
200
+
you can run the kernel and display the test results with the ``exec``
201
+
argument: ``./tools/testing/kunit/kunit.py exec``.
202
+
- ``parse`` extracts the KTAP output from a kernel log, parses
203
+
the test results, and prints a summary. For failed tests, any
204
+
diagnostic output will be included.
+2
Documentation/dev-tools/kunit/index.rst
+2
Documentation/dev-tools/kunit/index.rst
···
9
9
:caption: Contents:
10
10
11
11
start
12
+
architecture
12
13
usage
13
14
kunit-tool
14
15
api/index
···
97
96
================
98
97
99
98
* Documentation/dev-tools/kunit/start.rst - for KUnit new users.
99
+
* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.
100
100
* Documentation/dev-tools/kunit/usage.rst - KUnit features.
101
101
* Documentation/dev-tools/kunit/tips.rst - best practices with
102
102
examples.
+81
Documentation/dev-tools/kunit/kunit_suitememorydiagram.svg
+81
Documentation/dev-tools/kunit/kunit_suitememorydiagram.svg
···
1
+
<?xml version="1.0" encoding="UTF-8"?>
2
+
<svg width="796.93" height="555.73" version="1.1" viewBox="0 0 796.93 555.73" xmlns="http://www.w3.org/2000/svg">
3
+
<g transform="translate(-13.724 -17.943)">
4
+
<g fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a">
5
+
<rect x="323.56" y="18.443" width="115.75" height="41.331"/>
6
+
<rect x="323.56" y="463.09" width="115.75" height="41.331"/>
7
+
<rect x="323.56" y="531.84" width="115.75" height="41.331"/>
8
+
<rect x="323.56" y="88.931" width="115.75" height="74.231"/>
9
+
</g>
10
+
<g>
11
+
<rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
12
+
<text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
13
+
</g>
14
+
<g transform="translate(0 -258.6)">
15
+
<rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
16
+
<text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
17
+
</g>
18
+
<g transform="translate(0 -217.27)">
19
+
<rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
20
+
<text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
21
+
</g>
22
+
<g transform="translate(0 -175.94)">
23
+
<rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
24
+
<text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
25
+
</g>
26
+
<g transform="translate(0 -134.61)">
27
+
<rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
28
+
<text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
29
+
</g>
30
+
<g transform="translate(0 -41.331)">
31
+
<rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
32
+
<text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
33
+
</g>
34
+
<g transform="translate(3.4459e-5 -.71088)">
35
+
<rect x="502.19" y="143.16" width="201.13" height="41.331" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
36
+
<text x="512.02319" y="168.02026" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="512.02319" y="168.02026" font-family="monospace">_kunit_suites_start</tspan></text>
37
+
</g>
38
+
<g transform="translate(3.0518e-5 -3.1753)">
39
+
<rect x="502.19" y="445.69" width="201.13" height="41.331" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
40
+
<text x="521.61694" y="470.54846" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="521.61694" y="470.54846" font-family="monospace">_kunit_suites_end</tspan></text>
41
+
</g>
42
+
<rect x="14.224" y="277.78" width="134.47" height="41.331" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
43
+
<text x="32.062176" y="304.41287" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="32.062176" y="304.41287" font-family="monospace">.init.data</tspan></text>
44
+
<g transform="translate(217.98 145.12)" stroke="#1a1a1a">
45
+
<circle cx="149.97" cy="373.01" r="3.4012"/>
46
+
<circle cx="163.46" cy="373.01" r="3.4012"/>
47
+
<circle cx="176.95" cy="373.01" r="3.4012"/>
48
+
</g>
49
+
<g transform="translate(217.98 -298.66)" stroke="#1a1a1a">
50
+
<circle cx="149.97" cy="373.01" r="3.4012"/>
51
+
<circle cx="163.46" cy="373.01" r="3.4012"/>
52
+
<circle cx="176.95" cy="373.01" r="3.4012"/>
53
+
</g>
54
+
<g stroke="#1a1a1a">
55
+
<rect x="323.56" y="328.49" width="115.75" height="51.549" fill="#b9dbc6"/>
56
+
<g transform="translate(217.98 -18.75)">
57
+
<circle cx="149.97" cy="373.01" r="3.4012"/>
58
+
<circle cx="163.46" cy="373.01" r="3.4012"/>
59
+
<circle cx="176.95" cy="373.01" r="3.4012"/>
60
+
</g>
61
+
</g>
62
+
<g transform="scale(1.0933 .9147)" stroke-width="32.937" aria-label="{">
63
+
<path d="m275.49 545.57c-35.836-8.432-47.43-24.769-47.957-64.821v-88.536c-0.527-44.795-10.54-57.97-49.538-67.456 38.998-10.013 49.011-23.715 49.538-67.983v-88.536c0.527-40.052 12.121-56.389 47.957-64.821v-5.797c-65.348 0-85.901 17.391-86.955 73.253v93.806c-0.527 36.89-10.013 50.065-44.795 59.551 34.782 10.013 44.268 23.188 44.795 60.078v93.279c1.581 56.389 21.607 73.78 86.955 73.78z"/>
64
+
</g>
65
+
<g transform="scale(1.1071 .90325)" stroke-width="14.44" aria-label="{">
66
+
<path d="m461.46 443.55c-15.711-3.6967-20.794-10.859-21.025-28.418v-38.815c-0.23104-19.639-4.6209-25.415-21.718-29.574 17.097-4.3898 21.487-10.397 21.718-29.805v-38.815c0.23105-17.559 5.314-24.722 21.025-28.418v-2.5415c-28.649 0-37.66 7.6244-38.122 32.115v41.126c-0.23105 16.173-4.3898 21.949-19.639 26.108 15.249 4.3898 19.408 10.166 19.639 26.339v40.895c0.69313 24.722 9.4728 32.346 38.122 32.346z"/>
67
+
</g>
68
+
<path d="m449.55 161.84v2.5h49.504v-2.5z" color="#000000" style="-inkscape-stroke:none"/>
69
+
<g fill-rule="evenodd">
70
+
<path d="m443.78 163.09 8.65-5v10z" color="#000000" stroke-width="1pt" style="-inkscape-stroke:none"/>
71
+
<path d="m453.1 156.94-10.648 6.1543 0.99804 0.57812 9.6504 5.5781zm-1.334 2.3125v7.6856l-6.6504-3.8438z" color="#000000" style="-inkscape-stroke:none"/>
72
+
</g>
73
+
<path d="m449.55 461.91v2.5h49.504v-2.5z" color="#000000" style="-inkscape-stroke:none"/>
74
+
<g fill-rule="evenodd">
75
+
<path d="m443.78 463.16 8.65-5v10z" color="#000000" stroke-width="1pt" style="-inkscape-stroke:none"/>
76
+
<path d="m453.1 457-10.648 6.1562 0.99804 0.57617 9.6504 5.5781zm-1.334 2.3125v7.6856l-6.6504-3.8438z" color="#000000" style="-inkscape-stroke:none"/>
77
+
</g>
78
+
<rect x="515.64" y="223.9" width="294.52" height="178.49" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
79
+
<text x="523.33319" y="262.52542" font-family="monospace" font-size="14.667px" style="line-height:1.25" xml:space="preserve"><tspan x="523.33319" y="262.52542"><tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit_suite {</tspan><tspan x="523.33319" y="280.8588"><tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold"> const char</tspan> name[<tspan fill="#ff00ff" font-size="14.667px">256</tspan>];</tspan><tspan x="523.33319" y="299.19217"> <tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">int</tspan> (*init)(<tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit *);</tspan><tspan x="523.33319" y="317.52554"> <tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">void</tspan> (*exit)(<tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit *);</tspan><tspan x="523.33319" y="335.85892"> <tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit_case *test_cases;</tspan><tspan x="523.33319" y="354.19229"> ...</tspan><tspan x="523.33319" y="372.52567">};</tspan></text>
80
+
</g>
81
+
</svg>
+1
Documentation/dev-tools/kunit/start.rst
+1
Documentation/dev-tools/kunit/start.rst