Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

kunit: tool: Fix error messages for cases of no tests and wrong TAP header

This patch addresses misleading error messages reported by kunit_tool in
two cases. First, in the case of TAP output having an incorrect header
format or missing a header, the parser used to output an error message of
'no tests run!'. Now the parser outputs an error message of 'could not
parse test results!'.

As an example:

Before:
$ ./tools/testing/kunit/kunit.py parse /dev/null
[ERROR] no tests run!
...

After:
$ ./tools/testing/kunit/kunit.py parse /dev/null
[ERROR] could not parse test results!
...

Second, in the case of TAP output with the correct header but no
tests, the parser used to output an error message of 'could not parse
test results!'. Now the parser outputs an error message of 'no tests
run!'.

As an example:

Before:
$ echo -e 'TAP version 14\n1..0' | ./tools/testing/kunit/kunit.py parse
[ERROR] could not parse test results!

After:
$ echo -e 'TAP version 14\n1..0' | ./tools/testing/kunit/kunit.py parse
[ERROR] no tests run!

Additionally, this patch also corrects the tests in kunit_tool_test.py
and adds a test to check the error in the case of TAP output with the
correct header but no tests.

Signed-off-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

authored by

Rae Moar and committed by
Shuah Khan
ed01ad3a e73f0f0e

+19 -5
+4 -2
tools/testing/kunit/kunit_parser.py
··· 338 338 def parse_test_result(lines: LineStream) -> TestResult: 339 339 consume_non_diagnostic(lines) 340 340 if not lines or not parse_tap_header(lines): 341 - return TestResult(TestStatus.NO_TESTS, [], lines) 341 + return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) 342 342 expected_test_suite_num = parse_test_plan(lines) 343 - if not expected_test_suite_num: 343 + if expected_test_suite_num == 0: 344 + return TestResult(TestStatus.NO_TESTS, [], lines) 345 + elif expected_test_suite_num is None: 344 346 return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) 345 347 test_suites = [] 346 348 for i in range(1, expected_test_suite_num + 1):
+13 -3
tools/testing/kunit/kunit_tool_test.py
··· 157 157 kunit_parser.TestStatus.FAILURE, 158 158 result.status) 159 159 160 + def test_no_header(self): 161 + empty_log = test_data_path('test_is_test_passed-no_tests_run_no_header.log') 162 + with open(empty_log) as file: 163 + result = kunit_parser.parse_run_tests( 164 + kunit_parser.extract_tap_lines(file.readlines())) 165 + self.assertEqual(0, len(result.suites)) 166 + self.assertEqual( 167 + kunit_parser.TestStatus.FAILURE_TO_PARSE_TESTS, 168 + result.status) 169 + 160 170 def test_no_tests(self): 161 - empty_log = test_data_path('test_is_test_passed-no_tests_run.log') 171 + empty_log = test_data_path('test_is_test_passed-no_tests_run_with_header.log') 162 172 with open(empty_log) as file: 163 173 result = kunit_parser.parse_run_tests( 164 174 kunit_parser.extract_tap_lines(file.readlines())) ··· 183 173 with open(crash_log) as file: 184 174 result = kunit_parser.parse_run_tests( 185 175 kunit_parser.extract_tap_lines(file.readlines())) 186 - print_mock.assert_any_call(StrContains('no tests run!')) 176 + print_mock.assert_any_call(StrContains('could not parse test results!')) 187 177 print_mock.stop() 188 178 file.close() 189 179 ··· 319 309 result["sub_groups"][1]["test_cases"][0]) 320 310 321 311 def test_no_tests_json(self): 322 - result = self._json_for('test_is_test_passed-no_tests_run.log') 312 + result = self._json_for('test_is_test_passed-no_tests_run_with_header.log') 323 313 self.assertEqual(0, len(result['sub_groups'])) 324 314 325 315 class StrContains(str):
tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log tools/testing/kunit/test_data/test_is_test_passed-no_tests_run_no_header.log
+2
tools/testing/kunit/test_data/test_is_test_passed-no_tests_run_with_header.log
··· 1 + TAP version 14 2 + 1..0