Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Documentation: kunit: Add naming guidelines

As discussed in [1], KUnit tests have hitherto not had a particularly
consistent naming scheme. This adds documentation outlining how tests
and test suites should be named, including how those names should be
used in Kconfig entries and filenames.

[1]:
https://lore.kernel.org/linux-kselftest/202006141005.BA19A9D3@keescook/t/#u

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Tim Bird <tim.bird@sony.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

authored by

David Gow and committed by
Shuah Khan
2a41fc52 9123e3a7

+206
+1
Documentation/dev-tools/kunit/index.rst
··· 11 11 usage 12 12 kunit-tool 13 13 api/index 14 + style 14 15 faq 15 16 16 17 What is KUnit?
+205
Documentation/dev-tools/kunit/style.rst
··· 1 + .. SPDX-License-Identifier: GPL-2.0 2 + 3 + =========================== 4 + Test Style and Nomenclature 5 + =========================== 6 + 7 + To make finding, writing, and using KUnit tests as simple as possible, it's 8 + strongly encouraged that they are named and written according to the guidelines 9 + below. While it's possible to write KUnit tests which do not follow these rules, 10 + they may break some tooling, may conflict with other tests, and may not be run 11 + automatically by testing systems. 12 + 13 + It's recommended that you only deviate from these guidelines when: 14 + 15 + 1. Porting tests to KUnit which are already known with an existing name, or 16 + 2. Writing tests which would cause serious problems if automatically run (e.g., 17 + non-deterministically producing false positives or negatives, or taking an 18 + extremely long time to run). 19 + 20 + Subsystems, Suites, and Tests 21 + ============================= 22 + 23 + In order to make tests as easy to find as possible, they're grouped into suites 24 + and subsystems. A test suite is a group of tests which test a related area of 25 + the kernel, and a subsystem is a set of test suites which test different parts 26 + of the same kernel subsystem or driver. 27 + 28 + Subsystems 29 + ---------- 30 + 31 + Every test suite must belong to a subsystem. A subsystem is a collection of one 32 + or more KUnit test suites which test the same driver or part of the kernel. A 33 + rule of thumb is that a test subsystem should match a single kernel module. If 34 + the code being tested can't be compiled as a module, in many cases the subsystem 35 + should correspond to a directory in the source tree or an entry in the 36 + MAINTAINERS file. If unsure, follow the conventions set by tests in similar 37 + areas. 38 + 39 + Test subsystems should be named after the code being tested, either after the 40 + module (wherever possible), or after the directory or files being tested. Test 41 + subsystems should be named to avoid ambiguity where necessary. 42 + 43 + If a test subsystem name has multiple components, they should be separated by 44 + underscores. *Do not* include "test" or "kunit" directly in the subsystem name 45 + unless you are actually testing other tests or the kunit framework itself. 46 + 47 + Example subsystems could be: 48 + 49 + ``ext4`` 50 + Matches the module and filesystem name. 51 + ``apparmor`` 52 + Matches the module name and LSM name. 53 + ``kasan`` 54 + Common name for the tool, prominent part of the path ``mm/kasan`` 55 + ``snd_hda_codec_hdmi`` 56 + Has several components (``snd``, ``hda``, ``codec``, ``hdmi``) separated by 57 + underscores. Matches the module name. 58 + 59 + Avoid names like these: 60 + 61 + ``linear-ranges`` 62 + Names should use underscores, not dashes, to separate words. Prefer 63 + ``linear_ranges``. 64 + ``qos-kunit-test`` 65 + As well as using underscores, this name should not have "kunit-test" as a 66 + suffix, and ``qos`` is ambiguous as a subsystem name. ``power_qos`` would be a 67 + better name. 68 + ``pc_parallel_port`` 69 + The corresponding module name is ``parport_pc``, so this subsystem should also 70 + be named ``parport_pc``. 71 + 72 + .. note:: 73 + The KUnit API and tools do not explicitly know about subsystems. They're 74 + simply a way of categorising test suites and naming modules which 75 + provides a simple, consistent way for humans to find and run tests. This 76 + may change in the future, though. 77 + 78 + Suites 79 + ------ 80 + 81 + KUnit tests are grouped into test suites, which cover a specific area of 82 + functionality being tested. Test suites can have shared initialisation and 83 + shutdown code which is run for all tests in the suite. 84 + Not all subsystems will need to be split into multiple test suites (e.g. simple drivers). 85 + 86 + Test suites are named after the subsystem they are part of. If a subsystem 87 + contains several suites, the specific area under test should be appended to the 88 + subsystem name, separated by an underscore. 89 + 90 + In the event that there are multiple types of test using KUnit within a 91 + subsystem (e.g., both unit tests and integration tests), they should be put into 92 + separate suites, with the type of test as the last element in the suite name. 93 + Unless these tests are actually present, avoid using ``_test``, ``_unittest`` or 94 + similar in the suite name. 95 + 96 + The full test suite name (including the subsystem name) should be specified as 97 + the ``.name`` member of the ``kunit_suite`` struct, and forms the base for the 98 + module name (see below). 99 + 100 + Example test suites could include: 101 + 102 + ``ext4_inode`` 103 + Part of the ``ext4`` subsystem, testing the ``inode`` area. 104 + ``kunit_try_catch`` 105 + Part of the ``kunit`` implementation itself, testing the ``try_catch`` area. 106 + ``apparmor_property_entry`` 107 + Part of the ``apparmor`` subsystem, testing the ``property_entry`` area. 108 + ``kasan`` 109 + The ``kasan`` subsystem has only one suite, so the suite name is the same as 110 + the subsystem name. 111 + 112 + Avoid names like: 113 + 114 + ``ext4_ext4_inode`` 115 + There's no reason to state the subsystem twice. 116 + ``property_entry`` 117 + The suite name is ambiguous without the subsystem name. 118 + ``kasan_integration_test`` 119 + Because there is only one suite in the ``kasan`` subsystem, the suite should 120 + just be called ``kasan``. There's no need to redundantly add 121 + ``integration_test``. Should a separate test suite with, for example, unit 122 + tests be added, then that suite could be named ``kasan_unittest`` or similar. 123 + 124 + Test Cases 125 + ---------- 126 + 127 + Individual tests consist of a single function which tests a constrained 128 + codepath, property, or function. In the test output, individual tests' results 129 + will show up as subtests of the suite's results. 130 + 131 + Tests should be named after what they're testing. This is often the name of the 132 + function being tested, with a description of the input or codepath being tested. 133 + As tests are C functions, they should be named and written in accordance with 134 + the kernel coding style. 135 + 136 + .. note:: 137 + As tests are themselves functions, their names cannot conflict with 138 + other C identifiers in the kernel. This may require some creative 139 + naming. It's a good idea to make your test functions `static` to avoid 140 + polluting the global namespace. 141 + 142 + Example test names include: 143 + 144 + ``unpack_u32_with_null_name`` 145 + Tests the ``unpack_u32`` function when a NULL name is passed in. 146 + ``test_list_splice`` 147 + Tests the ``list_splice`` macro. It has the prefix ``test_`` to avoid a 148 + name conflict with the macro itself. 149 + 150 + 151 + Should it be necessary to refer to a test outside the context of its test suite, 152 + the *fully-qualified* name of a test should be the suite name followed by the 153 + test name, separated by a colon (i.e. ``suite:test``). 154 + 155 + Test Kconfig Entries 156 + ==================== 157 + 158 + Every test suite should be tied to a Kconfig entry. 159 + 160 + This Kconfig entry must: 161 + 162 + * be named ``CONFIG_<name>_KUNIT_TEST``: where <name> is the name of the test 163 + suite. 164 + * be listed either alongside the config entries for the driver/subsystem being 165 + tested, or be under [Kernel Hacking]→[Kernel Testing and Coverage] 166 + * depend on ``CONFIG_KUNIT`` 167 + * be visible only if ``CONFIG_KUNIT_ALL_TESTS`` is not enabled. 168 + * have a default value of ``CONFIG_KUNIT_ALL_TESTS``. 169 + * have a brief description of KUnit in the help text 170 + 171 + Unless there's a specific reason not to (e.g. the test is unable to be built as 172 + a module), Kconfig entries for tests should be tristate. 173 + 174 + An example Kconfig entry: 175 + 176 + .. code-block:: none 177 + 178 + config FOO_KUNIT_TEST 179 + tristate "KUnit test for foo" if !KUNIT_ALL_TESTS 180 + depends on KUNIT 181 + default KUNIT_ALL_TESTS 182 + help 183 + This builds unit tests for foo. 184 + 185 + For more information on KUnit and unit tests in general, please refer 186 + to the KUnit documentation in Documentation/dev-tools/kunit 187 + 188 + If unsure, say N 189 + 190 + 191 + Test File and Module Names 192 + ========================== 193 + 194 + KUnit tests can often be compiled as a module. These modules should be named 195 + after the test suite, followed by ``_test``. If this is likely to conflict with 196 + non-KUnit tests, the suffix ``_kunit`` can also be used. 197 + 198 + The easiest way of achieving this is to name the file containing the test suite 199 + ``<suite>_test.c`` (or, as above, ``<suite>_kunit.c``). This file should be 200 + placed next to the code under test. 201 + 202 + If the suite name contains some or all of the name of the test's parent 203 + directory, it may make sense to modify the source filename to reduce redundancy. 204 + For example, a ``foo_firmware`` suite could be in the ``foo/firmware_test.c`` 205 + file.