Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Documentation/sphinx: fix kernel-doc decode for non-utf-8 locale

On python3, Popen() universal_newlines=True converts the subprocess
stdout to unicode text using a codec based on user preferences. Given
LANG indicating ascii and utf-8 stdout from the subprocess, you'd get:

WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno
../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't
decode byte 0xe2 in position 6368: ordinal not in range(128)

Fix this by dropping universal_newlines=True and replacing the implicit
LANG specific decode with an explicit utf-8 decode. This also gets rid
of the annoying conditional code for python 2 vs. 3.

Fixes: ba3501859354 ("Documentation/sphinx: fix kernel-doc extension on python3")
Reference: http://mid.mail-archive.com/54c23e8e-89c0-5cea-0dcc-e938952c5642@infradead.org
Reported-and-tested-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

authored by

Jani Nikula and committed by
Jonathan Corbet
86c0f046 a27bfcab

+3 -5
+3 -5
Documentation/sphinx/kerneldoc.py
··· 27 27 # Please make sure this works on both python2 and python3. 28 28 # 29 29 30 + import codecs 30 31 import os 31 32 import subprocess 32 33 import sys ··· 89 88 try: 90 89 env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd))) 91 90 92 - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) 91 + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) 93 92 out, err = p.communicate() 94 93 95 - # python2 needs conversion to unicode. 96 - # python3 with universal_newlines=True returns strings. 97 - if sys.version_info.major < 3: 98 - out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') 94 + out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8') 99 95 100 96 if p.returncode != 0: 101 97 sys.stderr.write(err)