Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

kunit: tool: stop using a shell to run kernel under QEMU

Note: this potentially breaks custom qemu_configs if people are using
them! But the fix for them is simple, don't specify multiple arguments
in one string and don't add on a redundant ''.

It feels a bit iffy to be using a shell in the first place.

There's the usual shenanigans where people could pass in arbitrary shell
commands via --kernel_arg (since we're just adding '' around the
kernel_cmdline) or via a custom qemu_config.
This isn't too much of a concern given the nature of this script (and
the qemu_config file is in python, you can do w/e you want already).

But it does have some other drawbacks.

One example of a kunit-specific pain point:
If the relevant qemu binary is missing, we get output like this:
> /bin/sh: line 1: qemu-system-aarch64: command not found
This in turn results in our KTAP parser complaining about
missing/invalid KTAP, but we don't directly show the error!
It's even more annoying to debug when you consider --raw_output only
shows KUnit output by default, i.e. you need --raw_output=all to see it.

Whereas directly invoking the binary, Python will raise a
FileNotFoundError for us, which is a noisier but more clear.

Making this change requires
* splitting parameters like ['-m 256'] into ['-m', '256'] in
kunit/qemu_configs/*.py
* change [''] to [] in kunit/qemu_configs/*.py since otherwise
QEMU fails w/ 'Device needs media, but drive is empty'
* dropping explicit quoting of the kernel cmdline
* using shlex.quote() when we print what command we're running
so the user can copy-paste and run it

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

authored by

Daniel Latypov and committed by
Shuah Khan
3f0a50f3 c2497643

+22 -20
+10 -8
tools/testing/kunit/kunit_kernel.py
··· 11 11 import logging 12 12 import subprocess 13 13 import os 14 + import shlex 14 15 import shutil 15 16 import signal 16 17 import threading ··· 119 118 '-nodefaults', 120 119 '-m', '1024', 121 120 '-kernel', kernel_path, 122 - '-append', '\'' + ' '.join(params + [self._kernel_command_line]) + '\'', 121 + '-append', ' '.join(params + [self._kernel_command_line]), 123 122 '-no-reboot', 124 123 '-nographic', 125 - '-serial stdio'] + self._extra_qemu_params 126 - print('Running tests with:\n$', ' '.join(qemu_command)) 127 - return subprocess.Popen(' '.join(qemu_command), 128 - stdin=subprocess.PIPE, 129 - stdout=subprocess.PIPE, 130 - stderr=subprocess.STDOUT, 131 - text=True, shell=True, errors='backslashreplace') 124 + '-serial', 'stdio'] + self._extra_qemu_params 125 + # Note: shlex.join() does what we want, but requires python 3.8+. 126 + print('Running tests with:\n$', ' '.join(shlex.quote(arg) for arg in qemu_command)) 127 + return subprocess.Popen(qemu_command, 128 + stdin=subprocess.PIPE, 129 + stdout=subprocess.PIPE, 130 + stderr=subprocess.STDOUT, 131 + text=True, errors='backslashreplace') 132 132 133 133 class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): 134 134 """An abstraction over command line operations performed on a source tree."""
+1 -1
tools/testing/kunit/qemu_configs/alpha.py
··· 7 7 qemu_arch='alpha', 8 8 kernel_path='arch/alpha/boot/vmlinux', 9 9 kernel_command_line='console=ttyS0', 10 - extra_qemu_params=['']) 10 + extra_qemu_params=[])
+1 -1
tools/testing/kunit/qemu_configs/arm.py
··· 10 10 qemu_arch='arm', 11 11 kernel_path='arch/arm/boot/zImage', 12 12 kernel_command_line='console=ttyAMA0', 13 - extra_qemu_params=['-machine virt']) 13 + extra_qemu_params=['-machine', 'virt'])
+1 -1
tools/testing/kunit/qemu_configs/arm64.py
··· 9 9 qemu_arch='aarch64', 10 10 kernel_path='arch/arm64/boot/Image.gz', 11 11 kernel_command_line='console=ttyAMA0', 12 - extra_qemu_params=['-machine virt', '-cpu cortex-a57']) 12 + extra_qemu_params=['-machine', 'virt', '-cpu', 'cortex-a57'])
+1 -1
tools/testing/kunit/qemu_configs/i386.py
··· 7 7 qemu_arch='x86_64', 8 8 kernel_path='arch/x86/boot/bzImage', 9 9 kernel_command_line='console=ttyS0', 10 - extra_qemu_params=['']) 10 + extra_qemu_params=[])
+1 -1
tools/testing/kunit/qemu_configs/powerpc.py
··· 9 9 qemu_arch='ppc64', 10 10 kernel_path='vmlinux', 11 11 kernel_command_line='console=ttyS0', 12 - extra_qemu_params=['-M pseries', '-cpu power8']) 12 + extra_qemu_params=['-M', 'pseries', '-cpu', 'power8'])
+3 -3
tools/testing/kunit/qemu_configs/riscv.py
··· 26 26 kernel_path='arch/riscv/boot/Image', 27 27 kernel_command_line='console=ttyS0', 28 28 extra_qemu_params=[ 29 - '-machine virt', 30 - '-cpu rv64', 31 - '-bios opensbi-riscv64-generic-fw_dynamic.bin']) 29 + '-machine', 'virt', 30 + '-cpu', 'rv64', 31 + '-bios', 'opensbi-riscv64-generic-fw_dynamic.bin'])
+2 -2
tools/testing/kunit/qemu_configs/s390.py
··· 10 10 kernel_path='arch/s390/boot/bzImage', 11 11 kernel_command_line='console=ttyS0', 12 12 extra_qemu_params=[ 13 - '-machine s390-ccw-virtio', 14 - '-cpu qemu',]) 13 + '-machine', 's390-ccw-virtio', 14 + '-cpu', 'qemu',])
+1 -1
tools/testing/kunit/qemu_configs/sparc.py
··· 7 7 qemu_arch='sparc', 8 8 kernel_path='arch/sparc/boot/zImage', 9 9 kernel_command_line='console=ttyS0 mem=256M', 10 - extra_qemu_params=['-m 256']) 10 + extra_qemu_params=['-m', '256'])
+1 -1
tools/testing/kunit/qemu_configs/x86_64.py
··· 7 7 qemu_arch='x86_64', 8 8 kernel_path='arch/x86/boot/bzImage', 9 9 kernel_command_line='console=ttyS0', 10 - extra_qemu_params=['']) 10 + extra_qemu_params=[])