Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

kunit: add ability to specify suite-level init and exit functions

KUnit has support for setup/cleanup logic for each test case in a suite.
But it lacks the ability to specify setup/cleanup for the entire suite
itself.

This can be used to do setup that is too expensive or cumbersome to do
for each test.
Or it can be used to do simpler things like log debug information after
the suite completes.
It's a fairly common feature, so the lack of it is noticeable.

Some examples in other frameworks and languages:
* https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass
* https://google.github.io/googletest/reference/testing.html#Test::SetUpTestSuite

Meta:
This is very similar to this patch here: https://lore.kernel.org/linux-kselftest/20210805043503.20252-3-bvanassche@acm.org/
The changes from that patch:
* pass in `struct kunit *` so users can do stuff like
`kunit_info(suite, "debug message")`
* makes sure the init failure is bubbled up as a failure
* updates kunit-example-test.c to use a suite init
* Updates kunit/usage.rst to mention the new support
* some minor cosmetic things
* use `suite_{init,exit}` instead of `{init/exit}_suite`
* make suite init error message more consistent w/ test init
* etc.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

authored by

Daniel Latypov and committed by
Shuah Khan
1cdba21d cae56e17

+47 -8
+11 -8
Documentation/dev-tools/kunit/usage.rst
··· 125 125 many similar tests. In order to reduce duplication in these closely related 126 126 tests, most unit testing frameworks (including KUnit) provide the concept of a 127 127 *test suite*. A test suite is a collection of test cases for a unit of code 128 - with a setup function that gets invoked before every test case and then a tear 129 - down function that gets invoked after every test case completes. For example: 128 + with optional setup and teardown functions that run before/after the whole 129 + suite and/or every test case. For example: 130 130 131 131 .. code-block:: c 132 132 ··· 141 141 .name = "example", 142 142 .init = example_test_init, 143 143 .exit = example_test_exit, 144 + .suite_init = example_suite_init, 145 + .suite_exit = example_suite_exit, 144 146 .test_cases = example_test_cases, 145 147 }; 146 148 kunit_test_suite(example_test_suite); 147 149 148 - In the above example, the test suite ``example_test_suite`` would run the test 149 - cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``. Each 150 - would have ``example_test_init`` called immediately before it and 151 - ``example_test_exit`` called immediately after it. 152 - ``kunit_test_suite(example_test_suite)`` registers the test suite with the 153 - KUnit test framework. 150 + In the above example, the test suite ``example_test_suite`` would first run 151 + ``example_suite_init``, then run the test cases ``example_test_foo``, 152 + ``example_test_bar``, and ``example_test_baz``. Each would have 153 + ``example_test_init`` called immediately before it and ``example_test_exit`` 154 + called immediately after it. Finally, ``example_suite_exit`` would be called 155 + after everything else. ``kunit_test_suite(example_test_suite)`` registers the 156 + test suite with the KUnit test framework. 154 157 155 158 .. note:: 156 159 A test case will only run if it is associated with a test suite.
+5
include/kunit/test.h
··· 153 153 * struct kunit_suite - describes a related collection of &struct kunit_case 154 154 * 155 155 * @name: the name of the test. Purely informational. 156 + * @suite_init: called once per test suite before the test cases. 157 + * @suite_exit: called once per test suite after all test cases. 156 158 * @init: called before every test case. 157 159 * @exit: called after every test case. 158 160 * @test_cases: a null terminated array of test cases. ··· 169 167 */ 170 168 struct kunit_suite { 171 169 const char name[256]; 170 + int (*suite_init)(struct kunit_suite *suite); 171 + void (*suite_exit)(struct kunit_suite *suite); 172 172 int (*init)(struct kunit *test); 173 173 void (*exit)(struct kunit *test); 174 174 struct kunit_case *test_cases; ··· 179 175 char status_comment[KUNIT_STATUS_COMMENT_SIZE]; 180 176 struct dentry *debugfs; 181 177 char *log; 178 + int suite_init_err; 182 179 }; 183 180 184 181 /**
+14
lib/kunit/kunit-example-test.c
··· 41 41 } 42 42 43 43 /* 44 + * This is run once before all test cases in the suite. 45 + * See the comment on example_test_suite for more information. 46 + */ 47 + static int example_test_init_suite(struct kunit_suite *suite) 48 + { 49 + kunit_info(suite, "initializing suite\n"); 50 + 51 + return 0; 52 + } 53 + 54 + /* 44 55 * This test should always be skipped. 45 56 */ 46 57 static void example_skip_test(struct kunit *test) ··· 153 142 * may be specified which runs after every test case and can be used to for 154 143 * cleanup. For clarity, running tests in a test suite would behave as follows: 155 144 * 145 + * suite.suite_init(suite); 156 146 * suite.init(test); 157 147 * suite.test_case[0](test); 158 148 * suite.exit(test); 159 149 * suite.init(test); 160 150 * suite.test_case[1](test); 161 151 * suite.exit(test); 152 + * suite.suite_exit(suite); 162 153 * ...; 163 154 */ 164 155 static struct kunit_suite example_test_suite = { 165 156 .name = "example", 166 157 .init = example_test_init, 158 + .suite_init = example_test_init_suite, 167 159 .test_cases = example_test_cases, 168 160 }; 169 161
+17
lib/kunit/test.c
··· 179 179 const struct kunit_case *test_case; 180 180 enum kunit_status status = KUNIT_SKIPPED; 181 181 182 + if (suite->suite_init_err) 183 + return KUNIT_FAILURE; 184 + 182 185 kunit_suite_for_each_test_case(suite, test_case) { 183 186 if (test_case->status == KUNIT_FAILURE) 184 187 return KUNIT_FAILURE; ··· 501 498 struct kunit_result_stats suite_stats = { 0 }; 502 499 struct kunit_result_stats total_stats = { 0 }; 503 500 501 + if (suite->suite_init) { 502 + suite->suite_init_err = suite->suite_init(suite); 503 + if (suite->suite_init_err) { 504 + kunit_err(suite, KUNIT_SUBTEST_INDENT 505 + "# failed to initialize (%d)", suite->suite_init_err); 506 + goto suite_end; 507 + } 508 + } 509 + 504 510 kunit_print_suite_start(suite); 505 511 506 512 kunit_suite_for_each_test_case(suite, test_case) { ··· 563 551 kunit_accumulate_stats(&total_stats, param_stats); 564 552 } 565 553 554 + if (suite->suite_exit) 555 + suite->suite_exit(suite); 556 + 566 557 kunit_print_suite_stats(suite, suite_stats, total_stats); 558 + suite_end: 567 559 kunit_print_suite_end(suite); 568 560 569 561 return 0; ··· 578 562 { 579 563 kunit_debugfs_create_suite(suite); 580 564 suite->status_comment[0] = '\0'; 565 + suite->suite_init_err = 0; 581 566 } 582 567 583 568 int __kunit_test_suites_init(struct kunit_suite * const * const suites)