python314: address CVE-2025-4517, CVE-2025-4330, CVE-2025-4138, CVE-2024-12718, CVE-2025-4435 (#413689)

authored by OTABI Tomoya and committed by GitHub bcf80324 551c0c45

+2058
+2055
pkgs/development/interpreters/python/cpython/3.14/CVE-2025-4517.patch
···
··· 1 + From 9e0ac76d96cf80b49055f6d6b9a6763fb9215c2a Mon Sep 17 00:00:00 2001 2 + From: =?UTF-8?q?=C5=81ukasz=20Langa?= <lukasz@langa.pl> 3 + Date: Tue, 3 Jun 2025 14:05:00 +0200 4 + Subject: [PATCH] [3.14] gh-135034: Normalize link targets in tarfile, add 5 + `os.path.realpath(strict='allow_missing')` (gh-135037) (gh-135065) 6 + MIME-Version: 1.0 7 + Content-Type: text/plain; charset=UTF-8 8 + Content-Transfer-Encoding: 8bit 9 + 10 + Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517. 11 + 12 + (cherry picked from commit 3612d8f51741b11f36f8fb0494d79086bac9390a) 13 + 14 + Signed-off-by: Łukasz Langa <lukasz@langa.pl> 15 + Co-authored-by: Petr Viktorin <encukou@gmail.com> 16 + Co-authored-by: Seth Michael Larson <seth@python.org> 17 + Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> 18 + Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> 19 + --- 20 + Doc/library/os.path.rst | 32 +- 21 + Doc/library/tarfile.rst | 20 ++ 22 + Doc/whatsnew/3.14.rst | 32 ++ 23 + Lib/genericpath.py | 11 +- 24 + Lib/ntpath.py | 38 ++- 25 + Lib/posixpath.py | 57 ++-- 26 + Lib/tarfile.py | 163 +++++++-- 27 + Lib/test/test_ntpath.py | 216 ++++++++++-- 28 + Lib/test/test_posixpath.py | 252 +++++++++++--- 29 + Lib/test/test_tarfile.py | 310 +++++++++++++++++- 30 + ...-06-02-11-32-23.gh-issue-135034.RLGjbp.rst | 6 + 31 + 11 files changed, 967 insertions(+), 170 deletions(-) 32 + create mode 100644 Misc/NEWS.d/next/Security/2025-06-02-11-32-23.gh-issue-135034.RLGjbp.rst 33 + 34 + diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst 35 + index ecbbc1d7605f9f..f72aee19d8f332 100644 36 + --- a/Doc/library/os.path.rst 37 + +++ b/Doc/library/os.path.rst 38 + @@ -408,9 +408,26 @@ the :mod:`glob` module.) 39 + system). On Windows, this function will also resolve MS-DOS (also called 8.3) 40 + style names such as ``C:\\PROGRA~1`` to ``C:\\Program Files``. 41 + 42 + - If a path doesn't exist or a symlink loop is encountered, and *strict* is 43 + - ``True``, :exc:`OSError` is raised. If *strict* is ``False`` these errors 44 + - are ignored, and so the result might be missing or otherwise inaccessible. 45 + + By default, the path is evaluated up to the first component that does not 46 + + exist, is a symlink loop, or whose evaluation raises :exc:`OSError`. 47 + + All such components are appended unchanged to the existing part of the path. 48 + + 49 + + Some errors that are handled this way include "access denied", "not a 50 + + directory", or "bad argument to internal function". Thus, the 51 + + resulting path may be missing or inaccessible, may still contain 52 + + links or loops, and may traverse non-directories. 53 + + 54 + + This behavior can be modified by keyword arguments: 55 + + 56 + + If *strict* is ``True``, the first error encountered when evaluating the path is 57 + + re-raised. 58 + + In particular, :exc:`FileNotFoundError` is raised if *path* does not exist, 59 + + or another :exc:`OSError` if it is otherwise inaccessible. 60 + + 61 + + If *strict* is :py:data:`os.path.ALLOW_MISSING`, errors other than 62 + + :exc:`FileNotFoundError` are re-raised (as with ``strict=True``). 63 + + Thus, the returned path will not contain any symbolic links, but the named 64 + + file and some of its parent directories may be missing. 65 + 66 + .. note:: 67 + This function emulates the operating system's procedure for making a path 68 + @@ -429,6 +446,15 @@ the :mod:`glob` module.) 69 + .. versionchanged:: 3.10 70 + The *strict* parameter was added. 71 + 72 + + .. versionchanged:: next 73 + + The :py:data:`~os.path.ALLOW_MISSING` value for the *strict* parameter 74 + + was added. 75 + + 76 + +.. data:: ALLOW_MISSING 77 + + 78 + + Special value used for the *strict* argument in :func:`realpath`. 79 + + 80 + + .. versionadded:: next 81 + 82 + .. function:: relpath(path, start=os.curdir) 83 + 84 + diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst 85 + index f9cb5495e60cd2..7cec108a5bd41d 100644 86 + --- a/Doc/library/tarfile.rst 87 + +++ b/Doc/library/tarfile.rst 88 + @@ -255,6 +255,15 @@ The :mod:`tarfile` module defines the following exceptions: 89 + Raised to refuse extracting a symbolic link pointing outside the destination 90 + directory. 91 + 92 + +.. exception:: LinkFallbackError 93 + + 94 + + Raised to refuse emulating a link (hard or symbolic) by extracting another 95 + + archive member, when that member would be rejected by the filter location. 96 + + The exception that was raised to reject the replacement member is available 97 + + as :attr:`!BaseException.__context__`. 98 + + 99 + + .. versionadded:: next 100 + + 101 + 102 + The following constants are available at the module level: 103 + 104 + @@ -1068,6 +1077,12 @@ reused in custom filters: 105 + Implements the ``'data'`` filter. 106 + In addition to what ``tar_filter`` does: 107 + 108 + + - Normalize link targets (:attr:`TarInfo.linkname`) using 109 + + :func:`os.path.normpath`. 110 + + Note that this removes internal ``..`` components, which may change the 111 + + meaning of the link if the path in :attr:`!TarInfo.linkname` traverses 112 + + symbolic links. 113 + + 114 + - :ref:`Refuse <tarfile-extraction-refuse>` to extract links (hard or soft) 115 + that link to absolute paths, or ones that link outside the destination. 116 + 117 + @@ -1099,6 +1114,10 @@ reused in custom filters: 118 + Note that this filter does not block *all* dangerous archive features. 119 + See :ref:`tarfile-further-verification` for details. 120 + 121 + + .. versionchanged:: next 122 + + 123 + + Link targets are now normalized. 124 + + 125 + 126 + .. _tarfile-extraction-refuse: 127 + 128 + @@ -1127,6 +1146,7 @@ Here is an incomplete list of things to consider: 129 + * Extract to a :func:`new temporary directory <tempfile.mkdtemp>` 130 + to prevent e.g. exploiting pre-existing links, and to make it easier to 131 + clean up after a failed extraction. 132 + +* Disallow symbolic links if you do not need the functionality. 133 + * When working with untrusted data, use external (e.g. OS-level) limits on 134 + disk, memory and CPU usage. 135 + * Check filenames against an allow-list of characters 136 + diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst 137 + index 561d1a8914b50c..1e6a3615963ff4 100644 138 + --- a/Doc/whatsnew/3.14.rst 139 + +++ b/Doc/whatsnew/3.14.rst 140 + @@ -1608,6 +1608,16 @@ os 141 + (Contributed by Cody Maloney in :gh:`129205`.) 142 + 143 + 144 + +os.path 145 + +------- 146 + + 147 + +* The *strict* parameter to :func:`os.path.realpath` accepts a new value, 148 + + :data:`os.path.ALLOW_MISSING`. 149 + + If used, errors other than :exc:`FileNotFoundError` will be re-raised; 150 + + the resulting path can be missing but it will be free of symlinks. 151 + + (Contributed by Petr Viktorin for :cve:`2025-4517`.) 152 + + 153 + + 154 + pathlib 155 + ------- 156 + 157 + @@ -1796,6 +1806,28 @@ sysconfig 158 + (Contributed by Xuehai Pan in :gh:`131799`.) 159 + 160 + 161 + +tarfile 162 + +------- 163 + + 164 + +* :func:`~tarfile.data_filter` now normalizes symbolic link targets in order to 165 + + avoid path traversal attacks. 166 + + (Contributed by Petr Viktorin in :gh:`127987` and :cve:`2025-4138`.) 167 + +* :func:`~tarfile.TarFile.extractall` now skips fixing up directory attributes 168 + + when a directory was removed or replaced by another kind of file. 169 + + (Contributed by Petr Viktorin in :gh:`127987` and :cve:`2024-12718`.) 170 + +* :func:`~tarfile.TarFile.extract` and :func:`~tarfile.TarFile.extractall` 171 + + now (re-)apply the extraction filter when substituting a link (hard or 172 + + symbolic) with a copy of another archive member, and when fixing up 173 + + directory attributes. 174 + + The former raises a new exception, :exc:`~tarfile.LinkFallbackError`. 175 + + (Contributed by Petr Viktorin for :cve:`2025-4330` and :cve:`2024-12718`.) 176 + +* :func:`~tarfile.TarFile.extract` and :func:`~tarfile.TarFile.extractall` 177 + + no longer extract rejected members when 178 + + :func:`~tarfile.TarFile.errorlevel` is zero. 179 + + (Contributed by Matt Prodani and Petr Viktorin in :gh:`112887` 180 + + and :cve:`2025-4435`.) 181 + + 182 + + 183 + threading 184 + --------- 185 + 186 + diff --git a/Lib/genericpath.py b/Lib/genericpath.py 187 + index ba7b0a13c7f81d..9363f564aab7a6 100644 188 + --- a/Lib/genericpath.py 189 + +++ b/Lib/genericpath.py 190 + @@ -8,7 +8,7 @@ 191 + 192 + __all__ = ['commonprefix', 'exists', 'getatime', 'getctime', 'getmtime', 193 + 'getsize', 'isdevdrive', 'isdir', 'isfile', 'isjunction', 'islink', 194 + - 'lexists', 'samefile', 'sameopenfile', 'samestat'] 195 + + 'lexists', 'samefile', 'sameopenfile', 'samestat', 'ALLOW_MISSING'] 196 + 197 + 198 + # Does a path exist? 199 + @@ -189,3 +189,12 @@ def _check_arg_types(funcname, *args): 200 + f'os.PathLike object, not {s.__class__.__name__!r}') from None 201 + if hasstr and hasbytes: 202 + raise TypeError("Can't mix strings and bytes in path components") from None 203 + + 204 + +# A singleton with a true boolean value. 205 + +@object.__new__ 206 + +class ALLOW_MISSING: 207 + + """Special value for use in realpath().""" 208 + + def __repr__(self): 209 + + return 'os.path.ALLOW_MISSING' 210 + + def __reduce__(self): 211 + + return self.__class__.__name__ 212 + diff --git a/Lib/ntpath.py b/Lib/ntpath.py 213 + index 52ff2af743af6c..9cdc16480f9afe 100644 214 + --- a/Lib/ntpath.py 215 + +++ b/Lib/ntpath.py 216 + @@ -29,7 +29,7 @@ 217 + "abspath","curdir","pardir","sep","pathsep","defpath","altsep", 218 + "extsep","devnull","realpath","supports_unicode_filenames","relpath", 219 + "samefile", "sameopenfile", "samestat", "commonpath", "isjunction", 220 + - "isdevdrive"] 221 + + "isdevdrive", "ALLOW_MISSING"] 222 + 223 + def _get_bothseps(path): 224 + if isinstance(path, bytes): 225 + @@ -601,9 +601,10 @@ def abspath(path): 226 + from nt import _findfirstfile, _getfinalpathname, readlink as _nt_readlink 227 + except ImportError: 228 + # realpath is a no-op on systems without _getfinalpathname support. 229 + - realpath = abspath 230 + + def realpath(path, *, strict=False): 231 + + return abspath(path) 232 + else: 233 + - def _readlink_deep(path): 234 + + def _readlink_deep(path, ignored_error=OSError): 235 + # These error codes indicate that we should stop reading links and 236 + # return the path we currently have. 237 + # 1: ERROR_INVALID_FUNCTION 238 + @@ -636,7 +637,7 @@ def _readlink_deep(path): 239 + path = old_path 240 + break 241 + path = normpath(join(dirname(old_path), path)) 242 + - except OSError as ex: 243 + + except ignored_error as ex: 244 + if ex.winerror in allowed_winerror: 245 + break 246 + raise 247 + @@ -645,7 +646,7 @@ def _readlink_deep(path): 248 + break 249 + return path 250 + 251 + - def _getfinalpathname_nonstrict(path): 252 + + def _getfinalpathname_nonstrict(path, ignored_error=OSError): 253 + # These error codes indicate that we should stop resolving the path 254 + # and return the value we currently have. 255 + # 1: ERROR_INVALID_FUNCTION 256 + @@ -673,17 +674,18 @@ def _getfinalpathname_nonstrict(path): 257 + try: 258 + path = _getfinalpathname(path) 259 + return join(path, tail) if tail else path 260 + - except OSError as ex: 261 + + except ignored_error as ex: 262 + if ex.winerror not in allowed_winerror: 263 + raise 264 + try: 265 + # The OS could not resolve this path fully, so we attempt 266 + # to follow the link ourselves. If we succeed, join the tail 267 + # and return. 268 + - new_path = _readlink_deep(path) 269 + + new_path = _readlink_deep(path, 270 + + ignored_error=ignored_error) 271 + if new_path != path: 272 + return join(new_path, tail) if tail else new_path 273 + - except OSError: 274 + + except ignored_error: 275 + # If we fail to readlink(), let's keep traversing 276 + pass 277 + # If we get these errors, try to get the real name of the file without accessing it. 278 + @@ -691,7 +693,7 @@ def _getfinalpathname_nonstrict(path): 279 + try: 280 + name = _findfirstfile(path) 281 + path, _ = split(path) 282 + - except OSError: 283 + + except ignored_error: 284 + path, name = split(path) 285 + else: 286 + path, name = split(path) 287 + @@ -721,6 +723,15 @@ def realpath(path, *, strict=False): 288 + if normcase(path) == devnull: 289 + return '\\\\.\\NUL' 290 + had_prefix = path.startswith(prefix) 291 + + 292 + + if strict is ALLOW_MISSING: 293 + + ignored_error = FileNotFoundError 294 + + strict = True 295 + + elif strict: 296 + + ignored_error = () 297 + + else: 298 + + ignored_error = OSError 299 + + 300 + if not had_prefix and not isabs(path): 301 + path = join(cwd, path) 302 + try: 303 + @@ -728,17 +739,16 @@ def realpath(path, *, strict=False): 304 + initial_winerror = 0 305 + except ValueError as ex: 306 + # gh-106242: Raised for embedded null characters 307 + - # In strict mode, we convert into an OSError. 308 + + # In strict modes, we convert into an OSError. 309 + # Non-strict mode returns the path as-is, since we've already 310 + # made it absolute. 311 + if strict: 312 + raise OSError(str(ex)) from None 313 + path = normpath(path) 314 + - except OSError as ex: 315 + - if strict: 316 + - raise 317 + + except ignored_error as ex: 318 + initial_winerror = ex.winerror 319 + - path = _getfinalpathname_nonstrict(path) 320 + + path = _getfinalpathname_nonstrict(path, 321 + + ignored_error=ignored_error) 322 + # The path returned by _getfinalpathname will always start with \\?\ - 323 + # strip off that prefix unless it was already provided on the original 324 + # path. 325 + diff --git a/Lib/posixpath.py b/Lib/posixpath.py 326 + index db72ded8826056..d38f3bd5872bcd 100644 327 + --- a/Lib/posixpath.py 328 + +++ b/Lib/posixpath.py 329 + @@ -36,7 +36,7 @@ 330 + "samefile","sameopenfile","samestat", 331 + "curdir","pardir","sep","pathsep","defpath","altsep","extsep", 332 + "devnull","realpath","supports_unicode_filenames","relpath", 333 + - "commonpath", "isjunction","isdevdrive"] 334 + + "commonpath", "isjunction","isdevdrive","ALLOW_MISSING"] 335 + 336 + 337 + def _get_sep(path): 338 + @@ -402,10 +402,18 @@ def realpath(filename, *, strict=False): 339 + curdir = '.' 340 + pardir = '..' 341 + getcwd = os.getcwd 342 + - return _realpath(filename, strict, sep, curdir, pardir, getcwd) 343 + + if strict is ALLOW_MISSING: 344 + + ignored_error = FileNotFoundError 345 + + strict = True 346 + + elif strict: 347 + + ignored_error = () 348 + + else: 349 + + ignored_error = OSError 350 + + 351 + + lstat = os.lstat 352 + + readlink = os.readlink 353 + + maxlinks = None 354 + 355 + -def _realpath(filename, strict=False, sep=sep, curdir=curdir, pardir=pardir, 356 + - getcwd=os.getcwd, lstat=os.lstat, readlink=os.readlink, maxlinks=None): 357 + # The stack of unresolved path parts. When popped, a special value of None 358 + # indicates that a symlink target has been resolved, and that the original 359 + # symlink path can be retrieved by popping again. The [::-1] slice is a 360 + @@ -477,27 +485,28 @@ def _realpath(filename, strict=False, sep=sep, curdir=curdir, pardir=pardir, 361 + path = newpath 362 + continue 363 + target = readlink(newpath) 364 + - except OSError: 365 + - if strict: 366 + - raise 367 + - path = newpath 368 + + except ignored_error: 369 + + pass 370 + + else: 371 + + # Resolve the symbolic link 372 + + if target.startswith(sep): 373 + + # Symlink target is absolute; reset resolved path. 374 + + path = sep 375 + + if maxlinks is None: 376 + + # Mark this symlink as seen but not fully resolved. 377 + + seen[newpath] = None 378 + + # Push the symlink path onto the stack, and signal its specialness 379 + + # by also pushing None. When these entries are popped, we'll 380 + + # record the fully-resolved symlink target in the 'seen' mapping. 381 + + rest.append(newpath) 382 + + rest.append(None) 383 + + # Push the unresolved symlink target parts onto the stack. 384 + + target_parts = target.split(sep)[::-1] 385 + + rest.extend(target_parts) 386 + + part_count += len(target_parts) 387 + continue 388 + - # Resolve the symbolic link 389 + - if target.startswith(sep): 390 + - # Symlink target is absolute; reset resolved path. 391 + - path = sep 392 + - if maxlinks is None: 393 + - # Mark this symlink as seen but not fully resolved. 394 + - seen[newpath] = None 395 + - # Push the symlink path onto the stack, and signal its specialness 396 + - # by also pushing None. When these entries are popped, we'll 397 + - # record the fully-resolved symlink target in the 'seen' mapping. 398 + - rest.append(newpath) 399 + - rest.append(None) 400 + - # Push the unresolved symlink target parts onto the stack. 401 + - target_parts = target.split(sep)[::-1] 402 + - rest.extend(target_parts) 403 + - part_count += len(target_parts) 404 + + # An error occurred and was ignored. 405 + + path = newpath 406 + 407 + return path 408 + 409 + diff --git a/Lib/tarfile.py b/Lib/tarfile.py 410 + index 212b71f6509740..068aa13ed70356 100644 411 + --- a/Lib/tarfile.py 412 + +++ b/Lib/tarfile.py 413 + @@ -67,7 +67,7 @@ 414 + "DEFAULT_FORMAT", "open","fully_trusted_filter", "data_filter", 415 + "tar_filter", "FilterError", "AbsoluteLinkError", 416 + "OutsideDestinationError", "SpecialFileError", "AbsolutePathError", 417 + - "LinkOutsideDestinationError"] 418 + + "LinkOutsideDestinationError", "LinkFallbackError"] 419 + 420 + 421 + #--------------------------------------------------------- 422 + @@ -766,10 +766,22 @@ def __init__(self, tarinfo, path): 423 + super().__init__(f'{tarinfo.name!r} would link to {path!r}, ' 424 + + 'which is outside the destination') 425 + 426 + +class LinkFallbackError(FilterError): 427 + + def __init__(self, tarinfo, path): 428 + + self.tarinfo = tarinfo 429 + + self._path = path 430 + + super().__init__(f'link {tarinfo.name!r} would be extracted as a ' 431 + + + f'copy of {path!r}, which was rejected') 432 + + 433 + +# Errors caused by filters -- both "fatal" and "non-fatal" -- that 434 + +# we consider to be issues with the argument, rather than a bug in the 435 + +# filter function 436 + +_FILTER_ERRORS = (FilterError, OSError, ExtractError) 437 + + 438 + def _get_filtered_attrs(member, dest_path, for_data=True): 439 + new_attrs = {} 440 + name = member.name 441 + - dest_path = os.path.realpath(dest_path) 442 + + dest_path = os.path.realpath(dest_path, strict=os.path.ALLOW_MISSING) 443 + # Strip leading / (tar's directory separator) from filenames. 444 + # Include os.sep (target OS directory separator) as well. 445 + if name.startswith(('/', os.sep)): 446 + @@ -779,7 +791,8 @@ def _get_filtered_attrs(member, dest_path, for_data=True): 447 + # For example, 'C:/foo' on Windows. 448 + raise AbsolutePathError(member) 449 + # Ensure we stay in the destination 450 + - target_path = os.path.realpath(os.path.join(dest_path, name)) 451 + + target_path = os.path.realpath(os.path.join(dest_path, name), 452 + + strict=os.path.ALLOW_MISSING) 453 + if os.path.commonpath([target_path, dest_path]) != dest_path: 454 + raise OutsideDestinationError(member, target_path) 455 + # Limit permissions (no high bits, and go-w) 456 + @@ -817,6 +830,9 @@ def _get_filtered_attrs(member, dest_path, for_data=True): 457 + if member.islnk() or member.issym(): 458 + if os.path.isabs(member.linkname): 459 + raise AbsoluteLinkError(member) 460 + + normalized = os.path.normpath(member.linkname) 461 + + if normalized != member.linkname: 462 + + new_attrs['linkname'] = normalized 463 + if member.issym(): 464 + target_path = os.path.join(dest_path, 465 + os.path.dirname(name), 466 + @@ -824,7 +840,8 @@ def _get_filtered_attrs(member, dest_path, for_data=True): 467 + else: 468 + target_path = os.path.join(dest_path, 469 + member.linkname) 470 + - target_path = os.path.realpath(target_path) 471 + + target_path = os.path.realpath(target_path, 472 + + strict=os.path.ALLOW_MISSING) 473 + if os.path.commonpath([target_path, dest_path]) != dest_path: 474 + raise LinkOutsideDestinationError(member, target_path) 475 + return new_attrs 476 + @@ -2386,30 +2403,58 @@ def extractall(self, path=".", members=None, *, numeric_owner=False, 477 + members = self 478 + 479 + for member in members: 480 + - tarinfo = self._get_extract_tarinfo(member, filter_function, path) 481 + + tarinfo, unfiltered = self._get_extract_tarinfo( 482 + + member, filter_function, path) 483 + if tarinfo is None: 484 + continue 485 + if tarinfo.isdir(): 486 + # For directories, delay setting attributes until later, 487 + # since permissions can interfere with extraction and 488 + # extracting contents can reset mtime. 489 + - directories.append(tarinfo) 490 + + directories.append(unfiltered) 491 + self._extract_one(tarinfo, path, set_attrs=not tarinfo.isdir(), 492 + - numeric_owner=numeric_owner) 493 + + numeric_owner=numeric_owner, 494 + + filter_function=filter_function) 495 + 496 + # Reverse sort directories. 497 + directories.sort(key=lambda a: a.name, reverse=True) 498 + 499 + + 500 + # Set correct owner, mtime and filemode on directories. 501 + - for tarinfo in directories: 502 + - dirpath = os.path.join(path, tarinfo.name) 503 + + for unfiltered in directories: 504 + try: 505 + + # Need to re-apply any filter, to take the *current* filesystem 506 + + # state into account. 507 + + try: 508 + + tarinfo = filter_function(unfiltered, path) 509 + + except _FILTER_ERRORS as exc: 510 + + self._log_no_directory_fixup(unfiltered, repr(exc)) 511 + + continue 512 + + if tarinfo is None: 513 + + self._log_no_directory_fixup(unfiltered, 514 + + 'excluded by filter') 515 + + continue 516 + + dirpath = os.path.join(path, tarinfo.name) 517 + + try: 518 + + lstat = os.lstat(dirpath) 519 + + except FileNotFoundError: 520 + + self._log_no_directory_fixup(tarinfo, 'missing') 521 + + continue 522 + + if not stat.S_ISDIR(lstat.st_mode): 523 + + # This is no longer a directory; presumably a later 524 + + # member overwrote the entry. 525 + + self._log_no_directory_fixup(tarinfo, 'not a directory') 526 + + continue 527 + self.chown(tarinfo, dirpath, numeric_owner=numeric_owner) 528 + self.utime(tarinfo, dirpath) 529 + self.chmod(tarinfo, dirpath) 530 + except ExtractError as e: 531 + self._handle_nonfatal_error(e) 532 + 533 + + def _log_no_directory_fixup(self, member, reason): 534 + + self._dbg(2, "tarfile: Not fixing up directory %r (%s)" % 535 + + (member.name, reason)) 536 + + 537 + def extract(self, member, path="", set_attrs=True, *, numeric_owner=False, 538 + filter=None): 539 + """Extract a member from the archive to the current working directory, 540 + @@ -2425,41 +2470,56 @@ def extract(self, member, path="", set_attrs=True, *, numeric_owner=False, 541 + String names of common filters are accepted. 542 + """ 543 + filter_function = self._get_filter_function(filter) 544 + - tarinfo = self._get_extract_tarinfo(member, filter_function, path) 545 + + tarinfo, unfiltered = self._get_extract_tarinfo( 546 + + member, filter_function, path) 547 + if tarinfo is not None: 548 + self._extract_one(tarinfo, path, set_attrs, numeric_owner) 549 + 550 + def _get_extract_tarinfo(self, member, filter_function, path): 551 + - """Get filtered TarInfo (or None) from member, which might be a str""" 552 + + """Get (filtered, unfiltered) TarInfos from *member* 553 + + 554 + + *member* might be a string. 555 + + 556 + + Return (None, None) if not found. 557 + + """ 558 + + 559 + if isinstance(member, str): 560 + - tarinfo = self.getmember(member) 561 + + unfiltered = self.getmember(member) 562 + else: 563 + - tarinfo = member 564 + + unfiltered = member 565 + 566 + - unfiltered = tarinfo 567 + + filtered = None 568 + try: 569 + - tarinfo = filter_function(tarinfo, path) 570 + + filtered = filter_function(unfiltered, path) 571 + except (OSError, UnicodeEncodeError, FilterError) as e: 572 + self._handle_fatal_error(e) 573 + except ExtractError as e: 574 + self._handle_nonfatal_error(e) 575 + - if tarinfo is None: 576 + + if filtered is None: 577 + self._dbg(2, "tarfile: Excluded %r" % unfiltered.name) 578 + - return None 579 + + return None, None 580 + + 581 + # Prepare the link target for makelink(). 582 + - if tarinfo.islnk(): 583 + - tarinfo = copy.copy(tarinfo) 584 + - tarinfo._link_target = os.path.join(path, tarinfo.linkname) 585 + - return tarinfo 586 + + if filtered.islnk(): 587 + + filtered = copy.copy(filtered) 588 + + filtered._link_target = os.path.join(path, filtered.linkname) 589 + + return filtered, unfiltered 590 + + 591 + + def _extract_one(self, tarinfo, path, set_attrs, numeric_owner, 592 + + filter_function=None): 593 + + """Extract from filtered tarinfo to disk. 594 + 595 + - def _extract_one(self, tarinfo, path, set_attrs, numeric_owner): 596 + - """Extract from filtered tarinfo to disk""" 597 + + filter_function is only used when extracting a *different* 598 + + member (e.g. as fallback to creating a symlink) 599 + + """ 600 + self._check("r") 601 + 602 + try: 603 + self._extract_member(tarinfo, os.path.join(path, tarinfo.name), 604 + set_attrs=set_attrs, 605 + - numeric_owner=numeric_owner) 606 + + numeric_owner=numeric_owner, 607 + + filter_function=filter_function, 608 + + extraction_root=path) 609 + except (OSError, UnicodeEncodeError) as e: 610 + self._handle_fatal_error(e) 611 + except ExtractError as e: 612 + @@ -2517,9 +2577,13 @@ def extractfile(self, member): 613 + return None 614 + 615 + def _extract_member(self, tarinfo, targetpath, set_attrs=True, 616 + - numeric_owner=False): 617 + - """Extract the TarInfo object tarinfo to a physical 618 + + numeric_owner=False, *, filter_function=None, 619 + + extraction_root=None): 620 + + """Extract the filtered TarInfo object tarinfo to a physical 621 + file called targetpath. 622 + + 623 + + filter_function is only used when extracting a *different* 624 + + member (e.g. as fallback to creating a symlink) 625 + """ 626 + # Fetch the TarInfo object for the given name 627 + # and build the destination pathname, replacing 628 + @@ -2548,7 +2612,10 @@ def _extract_member(self, tarinfo, targetpath, set_attrs=True, 629 + elif tarinfo.ischr() or tarinfo.isblk(): 630 + self.makedev(tarinfo, targetpath) 631 + elif tarinfo.islnk() or tarinfo.issym(): 632 + - self.makelink(tarinfo, targetpath) 633 + + self.makelink_with_filter( 634 + + tarinfo, targetpath, 635 + + filter_function=filter_function, 636 + + extraction_root=extraction_root) 637 + elif tarinfo.type not in SUPPORTED_TYPES: 638 + self.makeunknown(tarinfo, targetpath) 639 + else: 640 + @@ -2631,10 +2698,18 @@ def makedev(self, tarinfo, targetpath): 641 + os.makedev(tarinfo.devmajor, tarinfo.devminor)) 642 + 643 + def makelink(self, tarinfo, targetpath): 644 + + return self.makelink_with_filter(tarinfo, targetpath, None, None) 645 + + 646 + + def makelink_with_filter(self, tarinfo, targetpath, 647 + + filter_function, extraction_root): 648 + """Make a (symbolic) link called targetpath. If it cannot be created 649 + (platform limitation), we try to make a copy of the referenced file 650 + instead of a link. 651 + + 652 + + filter_function is only used when extracting a *different* 653 + + member (e.g. as fallback to creating a link). 654 + """ 655 + + keyerror_to_extracterror = False 656 + try: 657 + # For systems that support symbolic and hard links. 658 + if tarinfo.issym(): 659 + @@ -2642,18 +2717,38 @@ def makelink(self, tarinfo, targetpath): 660 + # Avoid FileExistsError on following os.symlink. 661 + os.unlink(targetpath) 662 + os.symlink(tarinfo.linkname, targetpath) 663 + + return 664 + else: 665 + if os.path.exists(tarinfo._link_target): 666 + os.link(tarinfo._link_target, targetpath) 667 + - else: 668 + - self._extract_member(self._find_link_target(tarinfo), 669 + - targetpath) 670 + + return 671 + except symlink_exception: 672 + + keyerror_to_extracterror = True 673 + + 674 + + try: 675 + + unfiltered = self._find_link_target(tarinfo) 676 + + except KeyError: 677 + + if keyerror_to_extracterror: 678 + + raise ExtractError( 679 + + "unable to resolve link inside archive") from None 680 + + else: 681 + + raise 682 + + 683 + + if filter_function is None: 684 + + filtered = unfiltered 685 + + else: 686 + + if extraction_root is None: 687 + + raise ExtractError( 688 + + "makelink_with_filter: if filter_function is not None, " 689 + + + "extraction_root must also not be None") 690 + try: 691 + - self._extract_member(self._find_link_target(tarinfo), 692 + - targetpath) 693 + - except KeyError: 694 + - raise ExtractError("unable to resolve link inside archive") from None 695 + + filtered = filter_function(unfiltered, extraction_root) 696 + + except _FILTER_ERRORS as cause: 697 + + raise LinkFallbackError(tarinfo, unfiltered.name) from cause 698 + + if filtered is not None: 699 + + self._extract_member(filtered, targetpath, 700 + + filter_function=filter_function, 701 + + extraction_root=extraction_root) 702 + 703 + def chown(self, tarinfo, targetpath, numeric_owner): 704 + """Set owner of targetpath according to tarinfo. If numeric_owner 705 + diff --git a/Lib/test/test_ntpath.py b/Lib/test/test_ntpath.py 706 + index f83ef225a6e48e..927a0befcc8c13 100644 707 + --- a/Lib/test/test_ntpath.py 708 + +++ b/Lib/test/test_ntpath.py 709 + @@ -7,7 +7,8 @@ 710 + import unittest 711 + import warnings 712 + from test.support import cpython_only, os_helper 713 + -from test.support import TestFailed, is_emscripten 714 + +from test.support import TestFailed 715 + +from ntpath import ALLOW_MISSING 716 + from test.support.os_helper import FakePath 717 + from test import test_genericpath 718 + from tempfile import TemporaryFile 719 + @@ -77,6 +78,27 @@ def tester(fn, wantResult): 720 + %(str(fn), str(wantResult), repr(gotResult))) 721 + 722 + 723 + +def _parameterize(*parameters): 724 + + """Simplistic decorator to parametrize a test 725 + + 726 + + Runs the decorated test multiple times in subTest, with a value from 727 + + 'parameters' passed as an extra positional argument. 728 + + Calls doCleanups() after each run. 729 + + 730 + + Not for general use. Intended to avoid indenting for easier backports. 731 + + 732 + + See https://discuss.python.org/t/91827 for discussing generalizations. 733 + + """ 734 + + def _parametrize_decorator(func): 735 + + def _parameterized(self, *args, **kwargs): 736 + + for parameter in parameters: 737 + + with self.subTest(parameter): 738 + + func(self, *args, parameter, **kwargs) 739 + + self.doCleanups() 740 + + return _parameterized 741 + + return _parametrize_decorator 742 + + 743 + + 744 + class NtpathTestCase(unittest.TestCase): 745 + def assertPathEqual(self, path1, path2): 746 + if path1 == path2 or _norm(path1) == _norm(path2): 747 + @@ -475,6 +497,27 @@ def test_realpath_curdir(self): 748 + tester("ntpath.realpath('.\\.')", expected) 749 + tester("ntpath.realpath('\\'.join(['.'] * 100))", expected) 750 + 751 + + def test_realpath_curdir_strict(self): 752 + + expected = ntpath.normpath(os.getcwd()) 753 + + tester("ntpath.realpath('.', strict=True)", expected) 754 + + tester("ntpath.realpath('./.', strict=True)", expected) 755 + + tester("ntpath.realpath('/'.join(['.'] * 100), strict=True)", expected) 756 + + tester("ntpath.realpath('.\\.', strict=True)", expected) 757 + + tester("ntpath.realpath('\\'.join(['.'] * 100), strict=True)", expected) 758 + + 759 + + def test_realpath_curdir_missing_ok(self): 760 + + expected = ntpath.normpath(os.getcwd()) 761 + + tester("ntpath.realpath('.', strict=ALLOW_MISSING)", 762 + + expected) 763 + + tester("ntpath.realpath('./.', strict=ALLOW_MISSING)", 764 + + expected) 765 + + tester("ntpath.realpath('/'.join(['.'] * 100), strict=ALLOW_MISSING)", 766 + + expected) 767 + + tester("ntpath.realpath('.\\.', strict=ALLOW_MISSING)", 768 + + expected) 769 + + tester("ntpath.realpath('\\'.join(['.'] * 100), strict=ALLOW_MISSING)", 770 + + expected) 771 + + 772 + def test_realpath_pardir(self): 773 + expected = ntpath.normpath(os.getcwd()) 774 + tester("ntpath.realpath('..')", ntpath.dirname(expected)) 775 + @@ -487,24 +530,59 @@ def test_realpath_pardir(self): 776 + tester("ntpath.realpath('\\'.join(['..'] * 50))", 777 + ntpath.splitdrive(expected)[0] + '\\') 778 + 779 + + def test_realpath_pardir_strict(self): 780 + + expected = ntpath.normpath(os.getcwd()) 781 + + tester("ntpath.realpath('..', strict=True)", ntpath.dirname(expected)) 782 + + tester("ntpath.realpath('../..', strict=True)", 783 + + ntpath.dirname(ntpath.dirname(expected))) 784 + + tester("ntpath.realpath('/'.join(['..'] * 50), strict=True)", 785 + + ntpath.splitdrive(expected)[0] + '\\') 786 + + tester("ntpath.realpath('..\\..', strict=True)", 787 + + ntpath.dirname(ntpath.dirname(expected))) 788 + + tester("ntpath.realpath('\\'.join(['..'] * 50), strict=True)", 789 + + ntpath.splitdrive(expected)[0] + '\\') 790 + + 791 + + def test_realpath_pardir_missing_ok(self): 792 + + expected = ntpath.normpath(os.getcwd()) 793 + + tester("ntpath.realpath('..', strict=ALLOW_MISSING)", 794 + + ntpath.dirname(expected)) 795 + + tester("ntpath.realpath('../..', strict=ALLOW_MISSING)", 796 + + ntpath.dirname(ntpath.dirname(expected))) 797 + + tester("ntpath.realpath('/'.join(['..'] * 50), strict=ALLOW_MISSING)", 798 + + ntpath.splitdrive(expected)[0] + '\\') 799 + + tester("ntpath.realpath('..\\..', strict=ALLOW_MISSING)", 800 + + ntpath.dirname(ntpath.dirname(expected))) 801 + + tester("ntpath.realpath('\\'.join(['..'] * 50), strict=ALLOW_MISSING)", 802 + + ntpath.splitdrive(expected)[0] + '\\') 803 + + 804 + @os_helper.skip_unless_symlink 805 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 806 + - def test_realpath_basic(self): 807 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 808 + + def test_realpath_basic(self, kwargs): 809 + ABSTFN = ntpath.abspath(os_helper.TESTFN) 810 + open(ABSTFN, "wb").close() 811 + self.addCleanup(os_helper.unlink, ABSTFN) 812 + self.addCleanup(os_helper.unlink, ABSTFN + "1") 813 + 814 + os.symlink(ABSTFN, ABSTFN + "1") 815 + - self.assertPathEqual(ntpath.realpath(ABSTFN + "1"), ABSTFN) 816 + - self.assertPathEqual(ntpath.realpath(os.fsencode(ABSTFN + "1")), 817 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "1", **kwargs), ABSTFN) 818 + + self.assertPathEqual(ntpath.realpath(os.fsencode(ABSTFN + "1"), **kwargs), 819 + os.fsencode(ABSTFN)) 820 + 821 + # gh-88013: call ntpath.realpath with binary drive name may raise a 822 + # TypeError. The drive should not exist to reproduce the bug. 823 + drives = {f"{c}:\\" for c in string.ascii_uppercase} - set(os.listdrives()) 824 + d = drives.pop().encode() 825 + - self.assertEqual(ntpath.realpath(d), d) 826 + + self.assertEqual(ntpath.realpath(d, strict=False), d) 827 + + 828 + + # gh-106242: Embedded nulls and non-strict fallback to abspath 829 + + if kwargs: 830 + + with self.assertRaises(OSError): 831 + + ntpath.realpath(os_helper.TESTFN + "\0spam", 832 + + **kwargs) 833 + + else: 834 + + self.assertEqual(ABSTFN + "\0spam", 835 + + ntpath.realpath(os_helper.TESTFN + "\0spam", **kwargs)) 836 + 837 + @os_helper.skip_unless_symlink 838 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 839 + @@ -527,51 +605,66 @@ def test_realpath_invalid_paths(self): 840 + self.assertEqual(realpath(path, strict=False), path) 841 + # gh-106242: Embedded nulls should raise OSError (not ValueError) 842 + self.assertRaises(OSError, realpath, path, strict=True) 843 + + self.assertRaises(OSError, realpath, path, strict=ALLOW_MISSING) 844 + path = ABSTFNb + b'\x00' 845 + self.assertEqual(realpath(path, strict=False), path) 846 + self.assertRaises(OSError, realpath, path, strict=True) 847 + + self.assertRaises(OSError, realpath, path, strict=ALLOW_MISSING) 848 + path = ABSTFN + '\\nonexistent\\x\x00' 849 + self.assertEqual(realpath(path, strict=False), path) 850 + self.assertRaises(OSError, realpath, path, strict=True) 851 + + self.assertRaises(OSError, realpath, path, strict=ALLOW_MISSING) 852 + path = ABSTFNb + b'\\nonexistent\\x\x00' 853 + self.assertEqual(realpath(path, strict=False), path) 854 + self.assertRaises(OSError, realpath, path, strict=True) 855 + + self.assertRaises(OSError, realpath, path, strict=ALLOW_MISSING) 856 + path = ABSTFN + '\x00\\..' 857 + self.assertEqual(realpath(path, strict=False), os.getcwd()) 858 + self.assertEqual(realpath(path, strict=True), os.getcwd()) 859 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), os.getcwd()) 860 + path = ABSTFNb + b'\x00\\..' 861 + self.assertEqual(realpath(path, strict=False), os.getcwdb()) 862 + self.assertEqual(realpath(path, strict=True), os.getcwdb()) 863 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), os.getcwdb()) 864 + path = ABSTFN + '\\nonexistent\\x\x00\\..' 865 + self.assertEqual(realpath(path, strict=False), ABSTFN + '\\nonexistent') 866 + self.assertRaises(OSError, realpath, path, strict=True) 867 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), ABSTFN + '\\nonexistent') 868 + path = ABSTFNb + b'\\nonexistent\\x\x00\\..' 869 + self.assertEqual(realpath(path, strict=False), ABSTFNb + b'\\nonexistent') 870 + self.assertRaises(OSError, realpath, path, strict=True) 871 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), ABSTFNb + b'\\nonexistent') 872 + 873 + + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 874 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 875 + + def test_realpath_invalid_unicode_paths(self, kwargs): 876 + + realpath = ntpath.realpath 877 + + ABSTFN = ntpath.abspath(os_helper.TESTFN) 878 + + ABSTFNb = os.fsencode(ABSTFN) 879 + path = ABSTFNb + b'\xff' 880 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=False) 881 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=True) 882 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 883 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 884 + path = ABSTFNb + b'\\nonexistent\\\xff' 885 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=False) 886 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=True) 887 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 888 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 889 + path = ABSTFNb + b'\xff\\..' 890 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=False) 891 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=True) 892 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 893 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 894 + path = ABSTFNb + b'\\nonexistent\\\xff\\..' 895 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=False) 896 + - self.assertRaises(UnicodeDecodeError, realpath, path, strict=True) 897 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 898 + + self.assertRaises(UnicodeDecodeError, realpath, path, **kwargs) 899 + 900 + @os_helper.skip_unless_symlink 901 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 902 + - def test_realpath_relative(self): 903 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 904 + + def test_realpath_relative(self, kwargs): 905 + ABSTFN = ntpath.abspath(os_helper.TESTFN) 906 + open(ABSTFN, "wb").close() 907 + self.addCleanup(os_helper.unlink, ABSTFN) 908 + self.addCleanup(os_helper.unlink, ABSTFN + "1") 909 + 910 + os.symlink(ABSTFN, ntpath.relpath(ABSTFN + "1")) 911 + - self.assertPathEqual(ntpath.realpath(ABSTFN + "1"), ABSTFN) 912 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "1", **kwargs), ABSTFN) 913 + 914 + @os_helper.skip_unless_symlink 915 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 916 + @@ -723,7 +816,62 @@ def test_realpath_symlink_loops_strict(self): 917 + 918 + @os_helper.skip_unless_symlink 919 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 920 + - def test_realpath_symlink_prefix(self): 921 + + def test_realpath_symlink_loops_raise(self): 922 + + # Symlink loops raise OSError in ALLOW_MISSING mode 923 + + ABSTFN = ntpath.abspath(os_helper.TESTFN) 924 + + self.addCleanup(os_helper.unlink, ABSTFN) 925 + + self.addCleanup(os_helper.unlink, ABSTFN + "1") 926 + + self.addCleanup(os_helper.unlink, ABSTFN + "2") 927 + + self.addCleanup(os_helper.unlink, ABSTFN + "y") 928 + + self.addCleanup(os_helper.unlink, ABSTFN + "c") 929 + + self.addCleanup(os_helper.unlink, ABSTFN + "a") 930 + + self.addCleanup(os_helper.unlink, ABSTFN + "x") 931 + + 932 + + os.symlink(ABSTFN, ABSTFN) 933 + + self.assertRaises(OSError, ntpath.realpath, ABSTFN, strict=ALLOW_MISSING) 934 + + 935 + + os.symlink(ABSTFN + "1", ABSTFN + "2") 936 + + os.symlink(ABSTFN + "2", ABSTFN + "1") 937 + + self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1", 938 + + strict=ALLOW_MISSING) 939 + + self.assertRaises(OSError, ntpath.realpath, ABSTFN + "2", 940 + + strict=ALLOW_MISSING) 941 + + self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\x", 942 + + strict=ALLOW_MISSING) 943 + + 944 + + # Windows eliminates '..' components before resolving links; 945 + + # realpath is not expected to raise if this removes the loop. 946 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\.."), 947 + + ntpath.dirname(ABSTFN)) 948 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..\\x"), 949 + + ntpath.dirname(ABSTFN) + "\\x") 950 + + 951 + + os.symlink(ABSTFN + "x", ABSTFN + "y") 952 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..\\" 953 + + + ntpath.basename(ABSTFN) + "y"), 954 + + ABSTFN + "x") 955 + + self.assertRaises( 956 + + OSError, ntpath.realpath, 957 + + ABSTFN + "1\\..\\" + ntpath.basename(ABSTFN) + "1", 958 + + strict=ALLOW_MISSING) 959 + + 960 + + os.symlink(ntpath.basename(ABSTFN) + "a\\b", ABSTFN + "a") 961 + + self.assertRaises(OSError, ntpath.realpath, ABSTFN + "a", 962 + + strict=ALLOW_MISSING) 963 + + 964 + + os.symlink("..\\" + ntpath.basename(ntpath.dirname(ABSTFN)) 965 + + + "\\" + ntpath.basename(ABSTFN) + "c", ABSTFN + "c") 966 + + self.assertRaises(OSError, ntpath.realpath, ABSTFN + "c", 967 + + strict=ALLOW_MISSING) 968 + + 969 + + # Test using relative path as well. 970 + + self.assertRaises(OSError, ntpath.realpath, ntpath.basename(ABSTFN), 971 + + strict=ALLOW_MISSING) 972 + + 973 + + @os_helper.skip_unless_symlink 974 + + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 975 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 976 + + def test_realpath_symlink_prefix(self, kwargs): 977 + ABSTFN = ntpath.abspath(os_helper.TESTFN) 978 + self.addCleanup(os_helper.unlink, ABSTFN + "3") 979 + self.addCleanup(os_helper.unlink, "\\\\?\\" + ABSTFN + "3.") 980 + @@ -738,9 +886,9 @@ def test_realpath_symlink_prefix(self): 981 + f.write(b'1') 982 + os.symlink("\\\\?\\" + ABSTFN + "3.", ABSTFN + "3.link") 983 + 984 + - self.assertPathEqual(ntpath.realpath(ABSTFN + "3link"), 985 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "3link", **kwargs), 986 + ABSTFN + "3") 987 + - self.assertPathEqual(ntpath.realpath(ABSTFN + "3.link"), 988 + + self.assertPathEqual(ntpath.realpath(ABSTFN + "3.link", **kwargs), 989 + "\\\\?\\" + ABSTFN + "3.") 990 + 991 + # Resolved paths should be usable to open target files 992 + @@ -750,14 +898,17 @@ def test_realpath_symlink_prefix(self): 993 + self.assertEqual(f.read(), b'1') 994 + 995 + # When the prefix is included, it is not stripped 996 + - self.assertPathEqual(ntpath.realpath("\\\\?\\" + ABSTFN + "3link"), 997 + + self.assertPathEqual(ntpath.realpath("\\\\?\\" + ABSTFN + "3link", **kwargs), 998 + "\\\\?\\" + ABSTFN + "3") 999 + - self.assertPathEqual(ntpath.realpath("\\\\?\\" + ABSTFN + "3.link"), 1000 + + self.assertPathEqual(ntpath.realpath("\\\\?\\" + ABSTFN + "3.link", **kwargs), 1001 + "\\\\?\\" + ABSTFN + "3.") 1002 + 1003 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 1004 + def test_realpath_nul(self): 1005 + tester("ntpath.realpath('NUL')", r'\\.\NUL') 1006 + + tester("ntpath.realpath('NUL', strict=False)", r'\\.\NUL') 1007 + + tester("ntpath.realpath('NUL', strict=True)", r'\\.\NUL') 1008 + + tester("ntpath.realpath('NUL', strict=ALLOW_MISSING)", r'\\.\NUL') 1009 + 1010 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 1011 + @unittest.skipUnless(HAVE_GETSHORTPATHNAME, 'need _getshortpathname') 1012 + @@ -781,12 +932,20 @@ def test_realpath_cwd(self): 1013 + 1014 + self.assertPathEqual(test_file_long, ntpath.realpath(test_file_short)) 1015 + 1016 + - with os_helper.change_cwd(test_dir_long): 1017 + - self.assertPathEqual(test_file_long, ntpath.realpath("file.txt")) 1018 + - with os_helper.change_cwd(test_dir_long.lower()): 1019 + - self.assertPathEqual(test_file_long, ntpath.realpath("file.txt")) 1020 + - with os_helper.change_cwd(test_dir_short): 1021 + - self.assertPathEqual(test_file_long, ntpath.realpath("file.txt")) 1022 + + for kwargs in {}, {'strict': True}, {'strict': ALLOW_MISSING}: 1023 + + with self.subTest(**kwargs): 1024 + + with os_helper.change_cwd(test_dir_long): 1025 + + self.assertPathEqual( 1026 + + test_file_long, 1027 + + ntpath.realpath("file.txt", **kwargs)) 1028 + + with os_helper.change_cwd(test_dir_long.lower()): 1029 + + self.assertPathEqual( 1030 + + test_file_long, 1031 + + ntpath.realpath("file.txt", **kwargs)) 1032 + + with os_helper.change_cwd(test_dir_short): 1033 + + self.assertPathEqual( 1034 + + test_file_long, 1035 + + ntpath.realpath("file.txt", **kwargs)) 1036 + 1037 + @unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname') 1038 + def test_realpath_permission(self): 1039 + @@ -807,12 +966,15 @@ def test_realpath_permission(self): 1040 + # Automatic generation of short names may be disabled on 1041 + # NTFS volumes for the sake of performance. 1042 + # They're not supported at all on ReFS and exFAT. 1043 + - subprocess.run( 1044 + + p = subprocess.run( 1045 + # Try to set the short name manually. 1046 + ['fsutil.exe', 'file', 'setShortName', test_file, 'LONGFI~1.TXT'], 1047 + creationflags=subprocess.DETACHED_PROCESS 1048 + ) 1049 + 1050 + + if p.returncode: 1051 + + raise unittest.SkipTest('failed to set short name') 1052 + + 1053 + try: 1054 + self.assertPathEqual(test_file, ntpath.realpath(test_file_short)) 1055 + except AssertionError: 1056 + diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py 1057 + index f3f9895f529470..c45ce6d3ef7820 100644 1058 + --- a/Lib/test/test_posixpath.py 1059 + +++ b/Lib/test/test_posixpath.py 1060 + @@ -4,7 +4,8 @@ 1061 + import random 1062 + import sys 1063 + import unittest 1064 + -from posixpath import realpath, abspath, dirname, basename 1065 + +from functools import partial 1066 + +from posixpath import realpath, abspath, dirname, basename, ALLOW_MISSING 1067 + from test import support 1068 + from test import test_genericpath 1069 + from test.support import import_helper 1070 + @@ -33,6 +34,27 @@ def skip_if_ABSTFN_contains_backslash(test): 1071 + msg = "ABSTFN is not a posix path - tests fail" 1072 + return [test, unittest.skip(msg)(test)][found_backslash] 1073 + 1074 + + 1075 + +def _parameterize(*parameters): 1076 + + """Simplistic decorator to parametrize a test 1077 + + 1078 + + Runs the decorated test multiple times in subTest, with a value from 1079 + + 'parameters' passed as an extra positional argument. 1080 + + Does *not* call doCleanups() after each run. 1081 + + 1082 + + Not for general use. Intended to avoid indenting for easier backports. 1083 + + 1084 + + See https://discuss.python.org/t/91827 for discussing generalizations. 1085 + + """ 1086 + + def _parametrize_decorator(func): 1087 + + def _parameterized(self, *args, **kwargs): 1088 + + for parameter in parameters: 1089 + + with self.subTest(parameter): 1090 + + func(self, *args, parameter, **kwargs) 1091 + + return _parameterized 1092 + + return _parametrize_decorator 1093 + + 1094 + + 1095 + class PosixPathTest(unittest.TestCase): 1096 + 1097 + def setUp(self): 1098 + @@ -442,32 +464,35 @@ def test_normpath(self): 1099 + self.assertEqual(result, expected) 1100 + 1101 + @skip_if_ABSTFN_contains_backslash 1102 + - def test_realpath_curdir(self): 1103 + - self.assertEqual(realpath('.'), os.getcwd()) 1104 + - self.assertEqual(realpath('./.'), os.getcwd()) 1105 + - self.assertEqual(realpath('/'.join(['.'] * 100)), os.getcwd()) 1106 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1107 + + def test_realpath_curdir(self, kwargs): 1108 + + self.assertEqual(realpath('.', **kwargs), os.getcwd()) 1109 + + self.assertEqual(realpath('./.', **kwargs), os.getcwd()) 1110 + + self.assertEqual(realpath('/'.join(['.'] * 100), **kwargs), os.getcwd()) 1111 + 1112 + - self.assertEqual(realpath(b'.'), os.getcwdb()) 1113 + - self.assertEqual(realpath(b'./.'), os.getcwdb()) 1114 + - self.assertEqual(realpath(b'/'.join([b'.'] * 100)), os.getcwdb()) 1115 + + self.assertEqual(realpath(b'.', **kwargs), os.getcwdb()) 1116 + + self.assertEqual(realpath(b'./.', **kwargs), os.getcwdb()) 1117 + + self.assertEqual(realpath(b'/'.join([b'.'] * 100), **kwargs), os.getcwdb()) 1118 + 1119 + @skip_if_ABSTFN_contains_backslash 1120 + - def test_realpath_pardir(self): 1121 + - self.assertEqual(realpath('..'), dirname(os.getcwd())) 1122 + - self.assertEqual(realpath('../..'), dirname(dirname(os.getcwd()))) 1123 + - self.assertEqual(realpath('/'.join(['..'] * 100)), '/') 1124 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1125 + + def test_realpath_pardir(self, kwargs): 1126 + + self.assertEqual(realpath('..', **kwargs), dirname(os.getcwd())) 1127 + + self.assertEqual(realpath('../..', **kwargs), dirname(dirname(os.getcwd()))) 1128 + + self.assertEqual(realpath('/'.join(['..'] * 100), **kwargs), '/') 1129 + 1130 + - self.assertEqual(realpath(b'..'), dirname(os.getcwdb())) 1131 + - self.assertEqual(realpath(b'../..'), dirname(dirname(os.getcwdb()))) 1132 + - self.assertEqual(realpath(b'/'.join([b'..'] * 100)), b'/') 1133 + + self.assertEqual(realpath(b'..', **kwargs), dirname(os.getcwdb())) 1134 + + self.assertEqual(realpath(b'../..', **kwargs), dirname(dirname(os.getcwdb()))) 1135 + + self.assertEqual(realpath(b'/'.join([b'..'] * 100), **kwargs), b'/') 1136 + 1137 + @os_helper.skip_unless_symlink 1138 + @skip_if_ABSTFN_contains_backslash 1139 + - def test_realpath_basic(self): 1140 + + @_parameterize({}, {'strict': ALLOW_MISSING}) 1141 + + def test_realpath_basic(self, kwargs): 1142 + # Basic operation. 1143 + try: 1144 + os.symlink(ABSTFN+"1", ABSTFN) 1145 + - self.assertEqual(realpath(ABSTFN), ABSTFN+"1") 1146 + + self.assertEqual(realpath(ABSTFN, **kwargs), ABSTFN+"1") 1147 + finally: 1148 + os_helper.unlink(ABSTFN) 1149 + 1150 + @@ -487,90 +512,115 @@ def test_realpath_invalid_paths(self): 1151 + path = '/\x00' 1152 + self.assertRaises(ValueError, realpath, path, strict=False) 1153 + self.assertRaises(ValueError, realpath, path, strict=True) 1154 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1155 + path = b'/\x00' 1156 + self.assertRaises(ValueError, realpath, path, strict=False) 1157 + self.assertRaises(ValueError, realpath, path, strict=True) 1158 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1159 + path = '/nonexistent/x\x00' 1160 + self.assertRaises(ValueError, realpath, path, strict=False) 1161 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1162 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1163 + path = b'/nonexistent/x\x00' 1164 + self.assertRaises(ValueError, realpath, path, strict=False) 1165 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1166 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1167 + path = '/\x00/..' 1168 + self.assertRaises(ValueError, realpath, path, strict=False) 1169 + self.assertRaises(ValueError, realpath, path, strict=True) 1170 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1171 + path = b'/\x00/..' 1172 + self.assertRaises(ValueError, realpath, path, strict=False) 1173 + self.assertRaises(ValueError, realpath, path, strict=True) 1174 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1175 + + 1176 + path = '/nonexistent/x\x00/..' 1177 + self.assertRaises(ValueError, realpath, path, strict=False) 1178 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1179 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1180 + path = b'/nonexistent/x\x00/..' 1181 + self.assertRaises(ValueError, realpath, path, strict=False) 1182 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1183 + + self.assertRaises(ValueError, realpath, path, strict=ALLOW_MISSING) 1184 + 1185 + path = '/\udfff' 1186 + if sys.platform == 'win32': 1187 + self.assertEqual(realpath(path, strict=False), path) 1188 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1189 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), path) 1190 + else: 1191 + self.assertRaises(UnicodeEncodeError, realpath, path, strict=False) 1192 + self.assertRaises(UnicodeEncodeError, realpath, path, strict=True) 1193 + + self.assertRaises(UnicodeEncodeError, realpath, path, strict=ALLOW_MISSING) 1194 + path = '/nonexistent/\udfff' 1195 + if sys.platform == 'win32': 1196 + self.assertEqual(realpath(path, strict=False), path) 1197 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), path) 1198 + else: 1199 + self.assertRaises(UnicodeEncodeError, realpath, path, strict=False) 1200 + + self.assertRaises(UnicodeEncodeError, realpath, path, strict=ALLOW_MISSING) 1201 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1202 + path = '/\udfff/..' 1203 + if sys.platform == 'win32': 1204 + self.assertEqual(realpath(path, strict=False), '/') 1205 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1206 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), '/') 1207 + else: 1208 + self.assertRaises(UnicodeEncodeError, realpath, path, strict=False) 1209 + self.assertRaises(UnicodeEncodeError, realpath, path, strict=True) 1210 + + self.assertRaises(UnicodeEncodeError, realpath, path, strict=ALLOW_MISSING) 1211 + path = '/nonexistent/\udfff/..' 1212 + if sys.platform == 'win32': 1213 + self.assertEqual(realpath(path, strict=False), '/nonexistent') 1214 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), '/nonexistent') 1215 + else: 1216 + self.assertRaises(UnicodeEncodeError, realpath, path, strict=False) 1217 + + self.assertRaises(UnicodeEncodeError, realpath, path, strict=ALLOW_MISSING) 1218 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1219 + 1220 + path = b'/\xff' 1221 + if sys.platform == 'win32': 1222 + self.assertRaises(UnicodeDecodeError, realpath, path, strict=False) 1223 + self.assertRaises(UnicodeDecodeError, realpath, path, strict=True) 1224 + + self.assertRaises(UnicodeDecodeError, realpath, path, strict=ALLOW_MISSING) 1225 + else: 1226 + self.assertEqual(realpath(path, strict=False), path) 1227 + if support.is_wasi: 1228 + self.assertRaises(OSError, realpath, path, strict=True) 1229 + + self.assertRaises(OSError, realpath, path, strict=ALLOW_MISSING) 1230 + else: 1231 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1232 + + self.assertEqual(realpath(path, strict=ALLOW_MISSING), path) 1233 + path = b'/nonexistent/\xff' 1234 + if sys.platform == 'win32': 1235 + self.assertRaises(UnicodeDecodeError, realpath, path, strict=False) 1236 + + self.assertRaises(UnicodeDecodeError, realpath, path, strict=ALLOW_MISSING) 1237 + else: 1238 + self.assertEqual(realpath(path, strict=False), path) 1239 + if support.is_wasi: 1240 + self.assertRaises(OSError, realpath, path, strict=True) 1241 + + self.assertRaises(OSError, realpath, path, strict=ALLOW_MISSING) 1242 + else: 1243 + self.assertRaises(FileNotFoundError, realpath, path, strict=True) 1244 + 1245 + @os_helper.skip_unless_symlink 1246 + @skip_if_ABSTFN_contains_backslash 1247 + - def test_realpath_relative(self): 1248 + + @_parameterize({}, {'strict': ALLOW_MISSING}) 1249 + + def test_realpath_relative(self, kwargs): 1250 + try: 1251 + os.symlink(posixpath.relpath(ABSTFN+"1"), ABSTFN) 1252 + - self.assertEqual(realpath(ABSTFN), ABSTFN+"1") 1253 + + self.assertEqual(realpath(ABSTFN, **kwargs), ABSTFN+"1") 1254 + finally: 1255 + os_helper.unlink(ABSTFN) 1256 + 1257 + @os_helper.skip_unless_symlink 1258 + @skip_if_ABSTFN_contains_backslash 1259 + - def test_realpath_missing_pardir(self): 1260 + + @_parameterize({}, {'strict': ALLOW_MISSING}) 1261 + + def test_realpath_missing_pardir(self, kwargs): 1262 + try: 1263 + os.symlink(TESTFN + "1", TESTFN) 1264 + - self.assertEqual(realpath("nonexistent/../" + TESTFN), ABSTFN + "1") 1265 + + self.assertEqual( 1266 + + realpath("nonexistent/../" + TESTFN, **kwargs), ABSTFN + "1") 1267 + finally: 1268 + os_helper.unlink(TESTFN) 1269 + 1270 + @@ -617,37 +667,38 @@ def test_realpath_symlink_loops(self): 1271 + 1272 + @os_helper.skip_unless_symlink 1273 + @skip_if_ABSTFN_contains_backslash 1274 + - def test_realpath_symlink_loops_strict(self): 1275 + + @_parameterize({'strict': True}, {'strict': ALLOW_MISSING}) 1276 + + def test_realpath_symlink_loops_strict(self, kwargs): 1277 + # Bug #43757, raise OSError if we get into an infinite symlink loop in 1278 + - # strict mode. 1279 + + # the strict modes. 1280 + try: 1281 + os.symlink(ABSTFN, ABSTFN) 1282 + - self.assertRaises(OSError, realpath, ABSTFN, strict=True) 1283 + + self.assertRaises(OSError, realpath, ABSTFN, **kwargs) 1284 + 1285 + os.symlink(ABSTFN+"1", ABSTFN+"2") 1286 + os.symlink(ABSTFN+"2", ABSTFN+"1") 1287 + - self.assertRaises(OSError, realpath, ABSTFN+"1", strict=True) 1288 + - self.assertRaises(OSError, realpath, ABSTFN+"2", strict=True) 1289 + + self.assertRaises(OSError, realpath, ABSTFN+"1", **kwargs) 1290 + + self.assertRaises(OSError, realpath, ABSTFN+"2", **kwargs) 1291 + 1292 + - self.assertRaises(OSError, realpath, ABSTFN+"1/x", strict=True) 1293 + - self.assertRaises(OSError, realpath, ABSTFN+"1/..", strict=True) 1294 + - self.assertRaises(OSError, realpath, ABSTFN+"1/../x", strict=True) 1295 + + self.assertRaises(OSError, realpath, ABSTFN+"1/x", **kwargs) 1296 + + self.assertRaises(OSError, realpath, ABSTFN+"1/..", **kwargs) 1297 + + self.assertRaises(OSError, realpath, ABSTFN+"1/../x", **kwargs) 1298 + os.symlink(ABSTFN+"x", ABSTFN+"y") 1299 + self.assertRaises(OSError, realpath, 1300 + - ABSTFN+"1/../" + basename(ABSTFN) + "y", strict=True) 1301 + + ABSTFN+"1/../" + basename(ABSTFN) + "y", **kwargs) 1302 + self.assertRaises(OSError, realpath, 1303 + - ABSTFN+"1/../" + basename(ABSTFN) + "1", strict=True) 1304 + + ABSTFN+"1/../" + basename(ABSTFN) + "1", **kwargs) 1305 + 1306 + os.symlink(basename(ABSTFN) + "a/b", ABSTFN+"a") 1307 + - self.assertRaises(OSError, realpath, ABSTFN+"a", strict=True) 1308 + + self.assertRaises(OSError, realpath, ABSTFN+"a", **kwargs) 1309 + 1310 + os.symlink("../" + basename(dirname(ABSTFN)) + "/" + 1311 + basename(ABSTFN) + "c", ABSTFN+"c") 1312 + - self.assertRaises(OSError, realpath, ABSTFN+"c", strict=True) 1313 + + self.assertRaises(OSError, realpath, ABSTFN+"c", **kwargs) 1314 + 1315 + # Test using relative path as well. 1316 + with os_helper.change_cwd(dirname(ABSTFN)): 1317 + - self.assertRaises(OSError, realpath, basename(ABSTFN), strict=True) 1318 + + self.assertRaises(OSError, realpath, basename(ABSTFN), **kwargs) 1319 + finally: 1320 + os_helper.unlink(ABSTFN) 1321 + os_helper.unlink(ABSTFN+"1") 1322 + @@ -658,13 +709,14 @@ def test_realpath_symlink_loops_strict(self): 1323 + 1324 + @os_helper.skip_unless_symlink 1325 + @skip_if_ABSTFN_contains_backslash 1326 + - def test_realpath_repeated_indirect_symlinks(self): 1327 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1328 + + def test_realpath_repeated_indirect_symlinks(self, kwargs): 1329 + # Issue #6975. 1330 + try: 1331 + os.mkdir(ABSTFN) 1332 + os.symlink('../' + basename(ABSTFN), ABSTFN + '/self') 1333 + os.symlink('self/self/self', ABSTFN + '/link') 1334 + - self.assertEqual(realpath(ABSTFN + '/link'), ABSTFN) 1335 + + self.assertEqual(realpath(ABSTFN + '/link', **kwargs), ABSTFN) 1336 + finally: 1337 + os_helper.unlink(ABSTFN + '/self') 1338 + os_helper.unlink(ABSTFN + '/link') 1339 + @@ -672,14 +724,15 @@ def test_realpath_repeated_indirect_symlinks(self): 1340 + 1341 + @os_helper.skip_unless_symlink 1342 + @skip_if_ABSTFN_contains_backslash 1343 + - def test_realpath_deep_recursion(self): 1344 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1345 + + def test_realpath_deep_recursion(self, kwargs): 1346 + depth = 10 1347 + try: 1348 + os.mkdir(ABSTFN) 1349 + for i in range(depth): 1350 + os.symlink('/'.join(['%d' % i] * 10), ABSTFN + '/%d' % (i + 1)) 1351 + os.symlink('.', ABSTFN + '/0') 1352 + - self.assertEqual(realpath(ABSTFN + '/%d' % depth), ABSTFN) 1353 + + self.assertEqual(realpath(ABSTFN + '/%d' % depth, **kwargs), ABSTFN) 1354 + 1355 + # Test using relative path as well. 1356 + with os_helper.change_cwd(ABSTFN): 1357 + @@ -691,7 +744,8 @@ def test_realpath_deep_recursion(self): 1358 + 1359 + @os_helper.skip_unless_symlink 1360 + @skip_if_ABSTFN_contains_backslash 1361 + - def test_realpath_resolve_parents(self): 1362 + + @_parameterize({}, {'strict': ALLOW_MISSING}) 1363 + + def test_realpath_resolve_parents(self, kwargs): 1364 + # We also need to resolve any symlinks in the parents of a relative 1365 + # path passed to realpath. E.g.: current working directory is 1366 + # /usr/doc with 'doc' being a symlink to /usr/share/doc. We call 1367 + @@ -702,7 +756,8 @@ def test_realpath_resolve_parents(self): 1368 + os.symlink(ABSTFN + "/y", ABSTFN + "/k") 1369 + 1370 + with os_helper.change_cwd(ABSTFN + "/k"): 1371 + - self.assertEqual(realpath("a"), ABSTFN + "/y/a") 1372 + + self.assertEqual(realpath("a", **kwargs), 1373 + + ABSTFN + "/y/a") 1374 + finally: 1375 + os_helper.unlink(ABSTFN + "/k") 1376 + os_helper.rmdir(ABSTFN + "/y") 1377 + @@ -710,7 +765,8 @@ def test_realpath_resolve_parents(self): 1378 + 1379 + @os_helper.skip_unless_symlink 1380 + @skip_if_ABSTFN_contains_backslash 1381 + - def test_realpath_resolve_before_normalizing(self): 1382 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1383 + + def test_realpath_resolve_before_normalizing(self, kwargs): 1384 + # Bug #990669: Symbolic links should be resolved before we 1385 + # normalize the path. E.g.: if we have directories 'a', 'k' and 'y' 1386 + # in the following hierarchy: 1387 + @@ -725,10 +781,10 @@ def test_realpath_resolve_before_normalizing(self): 1388 + os.symlink(ABSTFN + "/k/y", ABSTFN + "/link-y") 1389 + 1390 + # Absolute path. 1391 + - self.assertEqual(realpath(ABSTFN + "/link-y/.."), ABSTFN + "/k") 1392 + + self.assertEqual(realpath(ABSTFN + "/link-y/..", **kwargs), ABSTFN + "/k") 1393 + # Relative path. 1394 + with os_helper.change_cwd(dirname(ABSTFN)): 1395 + - self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), 1396 + + self.assertEqual(realpath(basename(ABSTFN) + "/link-y/..", **kwargs), 1397 + ABSTFN + "/k") 1398 + finally: 1399 + os_helper.unlink(ABSTFN + "/link-y") 1400 + @@ -738,7 +794,8 @@ def test_realpath_resolve_before_normalizing(self): 1401 + 1402 + @os_helper.skip_unless_symlink 1403 + @skip_if_ABSTFN_contains_backslash 1404 + - def test_realpath_resolve_first(self): 1405 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1406 + + def test_realpath_resolve_first(self, kwargs): 1407 + # Bug #1213894: The first component of the path, if not absolute, 1408 + # must be resolved too. 1409 + 1410 + @@ -748,8 +805,8 @@ def test_realpath_resolve_first(self): 1411 + os.symlink(ABSTFN, ABSTFN + "link") 1412 + with os_helper.change_cwd(dirname(ABSTFN)): 1413 + base = basename(ABSTFN) 1414 + - self.assertEqual(realpath(base + "link"), ABSTFN) 1415 + - self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") 1416 + + self.assertEqual(realpath(base + "link", **kwargs), ABSTFN) 1417 + + self.assertEqual(realpath(base + "link/k", **kwargs), ABSTFN + "/k") 1418 + finally: 1419 + os_helper.unlink(ABSTFN + "link") 1420 + os_helper.rmdir(ABSTFN + "/k") 1421 + @@ -767,12 +824,67 @@ def test_realpath_unreadable_symlink(self): 1422 + self.assertEqual(realpath(ABSTFN + '/foo'), ABSTFN + '/foo') 1423 + self.assertEqual(realpath(ABSTFN + '/../foo'), dirname(ABSTFN) + '/foo') 1424 + self.assertEqual(realpath(ABSTFN + '/foo/..'), ABSTFN) 1425 + - with self.assertRaises(PermissionError): 1426 + - realpath(ABSTFN, strict=True) 1427 + finally: 1428 + os.chmod(ABSTFN, 0o755, follow_symlinks=False) 1429 + os_helper.unlink(ABSTFN) 1430 + 1431 + + @os_helper.skip_unless_symlink 1432 + + @skip_if_ABSTFN_contains_backslash 1433 + + @unittest.skipIf(os.chmod not in os.supports_follow_symlinks, "Can't set symlink permissions") 1434 + + @unittest.skipIf(sys.platform != "darwin", "only macOS requires read permission to readlink()") 1435 + + @_parameterize({'strict': True}, {'strict': ALLOW_MISSING}) 1436 + + def test_realpath_unreadable_symlink_strict(self, kwargs): 1437 + + try: 1438 + + os.symlink(ABSTFN+"1", ABSTFN) 1439 + + os.chmod(ABSTFN, 0o000, follow_symlinks=False) 1440 + + with self.assertRaises(PermissionError): 1441 + + realpath(ABSTFN, **kwargs) 1442 + + with self.assertRaises(PermissionError): 1443 + + realpath(ABSTFN + '/foo', **kwargs), 1444 + + with self.assertRaises(PermissionError): 1445 + + realpath(ABSTFN + '/../foo', **kwargs) 1446 + + with self.assertRaises(PermissionError): 1447 + + realpath(ABSTFN + '/foo/..', **kwargs) 1448 + + finally: 1449 + + os.chmod(ABSTFN, 0o755, follow_symlinks=False) 1450 + + os.unlink(ABSTFN) 1451 + + 1452 + + @skip_if_ABSTFN_contains_backslash 1453 + + @os_helper.skip_unless_symlink 1454 + + def test_realpath_unreadable_directory(self): 1455 + + try: 1456 + + os.mkdir(ABSTFN) 1457 + + os.mkdir(ABSTFN + '/k') 1458 + + os.chmod(ABSTFN, 0o000) 1459 + + self.assertEqual(realpath(ABSTFN, strict=False), ABSTFN) 1460 + + self.assertEqual(realpath(ABSTFN, strict=True), ABSTFN) 1461 + + self.assertEqual(realpath(ABSTFN, strict=ALLOW_MISSING), ABSTFN) 1462 + + 1463 + + try: 1464 + + os.stat(ABSTFN) 1465 + + except PermissionError: 1466 + + pass 1467 + + else: 1468 + + self.skipTest('Cannot block permissions') 1469 + + 1470 + + self.assertEqual(realpath(ABSTFN + '/k', strict=False), 1471 + + ABSTFN + '/k') 1472 + + self.assertRaises(PermissionError, realpath, ABSTFN + '/k', 1473 + + strict=True) 1474 + + self.assertRaises(PermissionError, realpath, ABSTFN + '/k', 1475 + + strict=ALLOW_MISSING) 1476 + + 1477 + + self.assertEqual(realpath(ABSTFN + '/missing', strict=False), 1478 + + ABSTFN + '/missing') 1479 + + self.assertRaises(PermissionError, realpath, ABSTFN + '/missing', 1480 + + strict=True) 1481 + + self.assertRaises(PermissionError, realpath, ABSTFN + '/missing', 1482 + + strict=ALLOW_MISSING) 1483 + + finally: 1484 + + os.chmod(ABSTFN, 0o755) 1485 + + os_helper.rmdir(ABSTFN + '/k') 1486 + + os_helper.rmdir(ABSTFN) 1487 + + 1488 + @skip_if_ABSTFN_contains_backslash 1489 + def test_realpath_nonterminal_file(self): 1490 + try: 1491 + @@ -780,14 +892,27 @@ def test_realpath_nonterminal_file(self): 1492 + f.write('test_posixpath wuz ere') 1493 + self.assertEqual(realpath(ABSTFN, strict=False), ABSTFN) 1494 + self.assertEqual(realpath(ABSTFN, strict=True), ABSTFN) 1495 + + self.assertEqual(realpath(ABSTFN, strict=ALLOW_MISSING), ABSTFN) 1496 + + 1497 + self.assertEqual(realpath(ABSTFN + "/", strict=False), ABSTFN) 1498 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/", strict=True) 1499 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/", 1500 + + strict=ALLOW_MISSING) 1501 + + 1502 + self.assertEqual(realpath(ABSTFN + "/.", strict=False), ABSTFN) 1503 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/.", strict=True) 1504 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/.", 1505 + + strict=ALLOW_MISSING) 1506 + + 1507 + self.assertEqual(realpath(ABSTFN + "/..", strict=False), dirname(ABSTFN)) 1508 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/..", strict=True) 1509 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/..", 1510 + + strict=ALLOW_MISSING) 1511 + + 1512 + self.assertEqual(realpath(ABSTFN + "/subdir", strict=False), ABSTFN + "/subdir") 1513 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/subdir", strict=True) 1514 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/subdir", 1515 + + strict=ALLOW_MISSING) 1516 + finally: 1517 + os_helper.unlink(ABSTFN) 1518 + 1519 + @@ -800,14 +925,27 @@ def test_realpath_nonterminal_symlink_to_file(self): 1520 + os.symlink(ABSTFN + "1", ABSTFN) 1521 + self.assertEqual(realpath(ABSTFN, strict=False), ABSTFN + "1") 1522 + self.assertEqual(realpath(ABSTFN, strict=True), ABSTFN + "1") 1523 + + self.assertEqual(realpath(ABSTFN, strict=ALLOW_MISSING), ABSTFN + "1") 1524 + + 1525 + self.assertEqual(realpath(ABSTFN + "/", strict=False), ABSTFN + "1") 1526 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/", strict=True) 1527 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/", 1528 + + strict=ALLOW_MISSING) 1529 + + 1530 + self.assertEqual(realpath(ABSTFN + "/.", strict=False), ABSTFN + "1") 1531 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/.", strict=True) 1532 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/.", 1533 + + strict=ALLOW_MISSING) 1534 + + 1535 + self.assertEqual(realpath(ABSTFN + "/..", strict=False), dirname(ABSTFN)) 1536 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/..", strict=True) 1537 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/..", 1538 + + strict=ALLOW_MISSING) 1539 + + 1540 + self.assertEqual(realpath(ABSTFN + "/subdir", strict=False), ABSTFN + "1/subdir") 1541 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/subdir", strict=True) 1542 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/subdir", 1543 + + strict=ALLOW_MISSING) 1544 + finally: 1545 + os_helper.unlink(ABSTFN) 1546 + os_helper.unlink(ABSTFN + "1") 1547 + @@ -822,14 +960,27 @@ def test_realpath_nonterminal_symlink_to_symlinks_to_file(self): 1548 + os.symlink(ABSTFN + "1", ABSTFN) 1549 + self.assertEqual(realpath(ABSTFN, strict=False), ABSTFN + "2") 1550 + self.assertEqual(realpath(ABSTFN, strict=True), ABSTFN + "2") 1551 + + self.assertEqual(realpath(ABSTFN, strict=True), ABSTFN + "2") 1552 + + 1553 + self.assertEqual(realpath(ABSTFN + "/", strict=False), ABSTFN + "2") 1554 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/", strict=True) 1555 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/", 1556 + + strict=ALLOW_MISSING) 1557 + + 1558 + self.assertEqual(realpath(ABSTFN + "/.", strict=False), ABSTFN + "2") 1559 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/.", strict=True) 1560 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/.", 1561 + + strict=ALLOW_MISSING) 1562 + + 1563 + self.assertEqual(realpath(ABSTFN + "/..", strict=False), dirname(ABSTFN)) 1564 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/..", strict=True) 1565 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/..", 1566 + + strict=ALLOW_MISSING) 1567 + + 1568 + self.assertEqual(realpath(ABSTFN + "/subdir", strict=False), ABSTFN + "2/subdir") 1569 + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/subdir", strict=True) 1570 + + self.assertRaises(NotADirectoryError, realpath, ABSTFN + "/subdir", 1571 + + strict=ALLOW_MISSING) 1572 + finally: 1573 + os_helper.unlink(ABSTFN) 1574 + os_helper.unlink(ABSTFN + "1") 1575 + @@ -1017,9 +1168,12 @@ def test_path_normpath(self): 1576 + def test_path_abspath(self): 1577 + self.assertPathEqual(self.path.abspath) 1578 + 1579 + - def test_path_realpath(self): 1580 + + @_parameterize({}, {'strict': True}, {'strict': ALLOW_MISSING}) 1581 + + def test_path_realpath(self, kwargs): 1582 + self.assertPathEqual(self.path.realpath) 1583 + 1584 + + self.assertPathEqual(partial(self.path.realpath, **kwargs)) 1585 + + 1586 + def test_path_relpath(self): 1587 + self.assertPathEqual(self.path.relpath) 1588 + 1589 + diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py 1590 + index cf218a2bf14369..7055e1ed147a9e 100644 1591 + --- a/Lib/test/test_tarfile.py 1592 + +++ b/Lib/test/test_tarfile.py 1593 + @@ -2715,6 +2715,31 @@ def test_useful_error_message_when_modules_missing(self): 1594 + str(excinfo.exception), 1595 + ) 1596 + 1597 + + @unittest.skipUnless(os_helper.can_symlink(), 'requires symlink support') 1598 + + @unittest.skipUnless(hasattr(os, 'chmod'), "missing os.chmod") 1599 + + @unittest.mock.patch('os.chmod') 1600 + + def test_deferred_directory_attributes_update(self, mock_chmod): 1601 + + # Regression test for gh-127987: setting attributes on arbitrary files 1602 + + tempdir = os.path.join(TEMPDIR, 'test127987') 1603 + + def mock_chmod_side_effect(path, mode, **kwargs): 1604 + + target_path = os.path.realpath(path) 1605 + + if os.path.commonpath([target_path, tempdir]) != tempdir: 1606 + + raise Exception("should not try to chmod anything outside the destination", target_path) 1607 + + mock_chmod.side_effect = mock_chmod_side_effect 1608 + + 1609 + + outside_tree_dir = os.path.join(TEMPDIR, 'outside_tree_dir') 1610 + + with ArchiveMaker() as arc: 1611 + + arc.add('x', symlink_to='.') 1612 + + arc.add('x', type=tarfile.DIRTYPE, mode='?rwsrwsrwt') 1613 + + arc.add('x', symlink_to=outside_tree_dir) 1614 + + 1615 + + os.makedirs(outside_tree_dir) 1616 + + try: 1617 + + arc.open().extractall(path=tempdir, filter='tar') 1618 + + finally: 1619 + + os_helper.rmtree(outside_tree_dir) 1620 + + os_helper.rmtree(tempdir) 1621 + + 1622 + 1623 + class CommandLineTest(unittest.TestCase): 1624 + 1625 + @@ -3275,6 +3300,10 @@ def check_files_present(self, directory): 1626 + got_paths = set( 1627 + p.relative_to(directory) 1628 + for p in pathlib.Path(directory).glob('**/*')) 1629 + + if self.extraction_filter in (None, 'data'): 1630 + + # The 'data' filter is expected to reject special files 1631 + + for path in 'ustar/fifotype', 'ustar/blktype', 'ustar/chrtype': 1632 + + got_paths.discard(pathlib.Path(path)) 1633 + self.assertEqual(self.control_paths, got_paths) 1634 + 1635 + @contextmanager 1636 + @@ -3504,12 +3533,28 @@ def __exit__(self, *exc): 1637 + self.bio = None 1638 + 1639 + def add(self, name, *, type=None, symlink_to=None, hardlink_to=None, 1640 + - mode=None, size=None, **kwargs): 1641 + - """Add a member to the test archive. Call within `with`.""" 1642 + + mode=None, size=None, content=None, **kwargs): 1643 + + """Add a member to the test archive. Call within `with`. 1644 + + 1645 + + Provides many shortcuts: 1646 + + - default `type` is based on symlink_to, hardlink_to, and trailing `/` 1647 + + in name (which is stripped) 1648 + + - size & content defaults are based on each other 1649 + + - content can be str or bytes 1650 + + - mode should be textual ('-rwxrwxrwx') 1651 + + 1652 + + (add more! this is unstable internal test-only API) 1653 + + """ 1654 + name = str(name) 1655 + tarinfo = tarfile.TarInfo(name).replace(**kwargs) 1656 + + if content is not None: 1657 + + if isinstance(content, str): 1658 + + content = content.encode() 1659 + + size = len(content) 1660 + if size is not None: 1661 + tarinfo.size = size 1662 + + if content is None: 1663 + + content = bytes(tarinfo.size) 1664 + if mode: 1665 + tarinfo.mode = _filemode_to_int(mode) 1666 + if symlink_to is not None: 1667 + @@ -3523,7 +3568,7 @@ def add(self, name, *, type=None, symlink_to=None, hardlink_to=None, 1668 + if type is not None: 1669 + tarinfo.type = type 1670 + if tarinfo.isreg(): 1671 + - fileobj = io.BytesIO(bytes(tarinfo.size)) 1672 + + fileobj = io.BytesIO(content) 1673 + else: 1674 + fileobj = None 1675 + self.tar_w.addfile(tarinfo, fileobj) 1676 + @@ -3557,7 +3602,7 @@ class TestExtractionFilters(unittest.TestCase): 1677 + destdir = outerdir / 'dest' 1678 + 1679 + @contextmanager 1680 + - def check_context(self, tar, filter): 1681 + + def check_context(self, tar, filter, *, check_flag=True): 1682 + """Extracts `tar` to `self.destdir` and allows checking the result 1683 + 1684 + If an error occurs, it must be checked using `expect_exception` 1685 + @@ -3566,27 +3611,40 @@ def check_context(self, tar, filter): 1686 + except the destination directory itself and parent directories of 1687 + other files. 1688 + When checking directories, do so before their contents. 1689 + + 1690 + + A file called 'flag' is made in outerdir (i.e. outside destdir) 1691 + + before extraction; it should not be altered nor should its contents 1692 + + be read/copied. 1693 + """ 1694 + with os_helper.temp_dir(self.outerdir): 1695 + + flag_path = self.outerdir / 'flag' 1696 + + flag_path.write_text('capture me') 1697 + try: 1698 + tar.extractall(self.destdir, filter=filter) 1699 + except Exception as exc: 1700 + self.raised_exception = exc 1701 + + self.reraise_exception = True 1702 + self.expected_paths = set() 1703 + else: 1704 + self.raised_exception = None 1705 + + self.reraise_exception = False 1706 + self.expected_paths = set(self.outerdir.glob('**/*')) 1707 + self.expected_paths.discard(self.destdir) 1708 + + self.expected_paths.discard(flag_path) 1709 + try: 1710 + - yield 1711 + + yield self 1712 + finally: 1713 + tar.close() 1714 + - if self.raised_exception: 1715 + + if self.reraise_exception: 1716 + raise self.raised_exception 1717 + self.assertEqual(self.expected_paths, set()) 1718 + + if check_flag: 1719 + + self.assertEqual(flag_path.read_text(), 'capture me') 1720 + + else: 1721 + + assert filter == 'fully_trusted' 1722 + 1723 + def expect_file(self, name, type=None, symlink_to=None, mode=None, 1724 + - size=None): 1725 + + size=None, content=None): 1726 + """Check a single file. See check_context.""" 1727 + if self.raised_exception: 1728 + raise self.raised_exception 1729 + @@ -3605,26 +3663,45 @@ def expect_file(self, name, type=None, symlink_to=None, mode=None, 1730 + # The symlink might be the same (textually) as what we expect, 1731 + # but some systems change the link to an equivalent path, so 1732 + # we fall back to samefile(). 1733 + - if expected != got: 1734 + - self.assertTrue(got.samefile(expected)) 1735 + + try: 1736 + + if expected != got: 1737 + + self.assertTrue(got.samefile(expected)) 1738 + + except Exception as e: 1739 + + # attach a note, so it's shown even if `samefile` fails 1740 + + e.add_note(f'{expected=}, {got=}') 1741 + + raise 1742 + elif type == tarfile.REGTYPE or type is None: 1743 + self.assertTrue(path.is_file()) 1744 + elif type == tarfile.DIRTYPE: 1745 + self.assertTrue(path.is_dir()) 1746 + elif type == tarfile.FIFOTYPE: 1747 + self.assertTrue(path.is_fifo()) 1748 + + elif type == tarfile.SYMTYPE: 1749 + + self.assertTrue(path.is_symlink()) 1750 + else: 1751 + raise NotImplementedError(type) 1752 + if size is not None: 1753 + self.assertEqual(path.stat().st_size, size) 1754 + + if content is not None: 1755 + + self.assertEqual(path.read_text(), content) 1756 + for parent in path.parents: 1757 + self.expected_paths.discard(parent) 1758 + 1759 + + def expect_any_tree(self, name): 1760 + + """Check a directory; forget about its contents.""" 1761 + + tree_path = (self.destdir / name).resolve() 1762 + + self.expect_file(tree_path, type=tarfile.DIRTYPE) 1763 + + self.expected_paths = { 1764 + + p for p in self.expected_paths 1765 + + if tree_path not in p.parents 1766 + + } 1767 + + 1768 + def expect_exception(self, exc_type, message_re='.'): 1769 + with self.assertRaisesRegex(exc_type, message_re): 1770 + if self.raised_exception is not None: 1771 + raise self.raised_exception 1772 + - self.raised_exception = None 1773 + + self.reraise_exception = False 1774 + + return self.raised_exception 1775 + 1776 + def test_benign_file(self): 1777 + with ArchiveMaker() as arc: 1778 + @@ -3709,6 +3786,80 @@ def test_parent_symlink(self): 1779 + with self.check_context(arc.open(), 'data'): 1780 + self.expect_file('parent/evil') 1781 + 1782 + + @symlink_test 1783 + + @os_helper.skip_unless_symlink 1784 + + def test_realpath_limit_attack(self): 1785 + + # (CVE-2025-4517) 1786 + + 1787 + + with ArchiveMaker() as arc: 1788 + + # populate the symlinks and dirs that expand in os.path.realpath() 1789 + + # The component length is chosen so that in common cases, the unexpanded 1790 + + # path fits in PATH_MAX, but it overflows when the final symlink 1791 + + # is expanded 1792 + + steps = "abcdefghijklmnop" 1793 + + if sys.platform == 'win32': 1794 + + component = 'd' * 25 1795 + + elif 'PC_PATH_MAX' in os.pathconf_names: 1796 + + max_path_len = os.pathconf(self.outerdir.parent, "PC_PATH_MAX") 1797 + + path_sep_len = 1 1798 + + dest_len = len(str(self.destdir)) + path_sep_len 1799 + + component_len = (max_path_len - dest_len) // (len(steps) + path_sep_len) 1800 + + component = 'd' * component_len 1801 + + else: 1802 + + raise NotImplementedError("Need to guess component length for {sys.platform}") 1803 + + path = "" 1804 + + step_path = "" 1805 + + for i in steps: 1806 + + arc.add(os.path.join(path, component), type=tarfile.DIRTYPE, 1807 + + mode='drwxrwxrwx') 1808 + + arc.add(os.path.join(path, i), symlink_to=component) 1809 + + path = os.path.join(path, component) 1810 + + step_path = os.path.join(step_path, i) 1811 + + # create the final symlink that exceeds PATH_MAX and simply points 1812 + + # to the top dir. 1813 + + # this link will never be expanded by 1814 + + # os.path.realpath(strict=False), nor anything after it. 1815 + + linkpath = os.path.join(*steps, "l"*254) 1816 + + parent_segments = [".."] * len(steps) 1817 + + arc.add(linkpath, symlink_to=os.path.join(*parent_segments)) 1818 + + # make a symlink outside to keep the tar command happy 1819 + + arc.add("escape", symlink_to=os.path.join(linkpath, "..")) 1820 + + # use the symlinks above, that are not checked, to create a hardlink 1821 + + # to a file outside of the destination path 1822 + + arc.add("flaglink", hardlink_to=os.path.join("escape", "flag")) 1823 + + # now that we have the hardlink we can overwrite the file 1824 + + arc.add("flaglink", content='overwrite') 1825 + + # we can also create new files as well! 1826 + + arc.add("escape/newfile", content='new') 1827 + + 1828 + + with (self.subTest('fully_trusted'), 1829 + + self.check_context(arc.open(), filter='fully_trusted', 1830 + + check_flag=False)): 1831 + + if sys.platform == 'win32': 1832 + + self.expect_exception((FileNotFoundError, FileExistsError)) 1833 + + elif self.raised_exception: 1834 + + # Cannot symlink/hardlink: tarfile falls back to getmember() 1835 + + self.expect_exception(KeyError) 1836 + + # Otherwise, this block should never enter. 1837 + + else: 1838 + + self.expect_any_tree(component) 1839 + + self.expect_file('flaglink', content='overwrite') 1840 + + self.expect_file('../newfile', content='new') 1841 + + self.expect_file('escape', type=tarfile.SYMTYPE) 1842 + + self.expect_file('a', symlink_to=component) 1843 + + 1844 + + for filter in 'tar', 'data': 1845 + + with self.subTest(filter), self.check_context(arc.open(), filter=filter): 1846 + + exc = self.expect_exception((OSError, KeyError)) 1847 + + if isinstance(exc, OSError): 1848 + + if sys.platform == 'win32': 1849 + + # 3: ERROR_PATH_NOT_FOUND 1850 + + # 5: ERROR_ACCESS_DENIED 1851 + + # 206: ERROR_FILENAME_EXCED_RANGE 1852 + + self.assertIn(exc.winerror, (3, 5, 206)) 1853 + + else: 1854 + + self.assertEqual(exc.errno, errno.ENAMETOOLONG) 1855 + + 1856 + @symlink_test 1857 + def test_parent_symlink2(self): 1858 + # Test interplaying symlinks 1859 + @@ -3931,8 +4082,8 @@ def test_chains(self): 1860 + arc.add('symlink2', symlink_to=os.path.join( 1861 + 'linkdir', 'hardlink2')) 1862 + arc.add('targetdir/target', size=3) 1863 + - arc.add('linkdir/hardlink', hardlink_to='targetdir/target') 1864 + - arc.add('linkdir/hardlink2', hardlink_to='linkdir/symlink') 1865 + + arc.add('linkdir/hardlink', hardlink_to=os.path.join('targetdir', 'target')) 1866 + + arc.add('linkdir/hardlink2', hardlink_to=os.path.join('linkdir', 'symlink')) 1867 + 1868 + for filter in 'tar', 'data', 'fully_trusted': 1869 + with self.check_context(arc.open(), filter): 1870 + @@ -3948,6 +4099,129 @@ def test_chains(self): 1871 + self.expect_file('linkdir/symlink', size=3) 1872 + self.expect_file('symlink2', size=3) 1873 + 1874 + + @symlink_test 1875 + + def test_sneaky_hardlink_fallback(self): 1876 + + # (CVE-2025-4330) 1877 + + # Test that when hardlink extraction falls back to extracting members 1878 + + # from the archive, the extracted member is (re-)filtered. 1879 + + with ArchiveMaker() as arc: 1880 + + # Create a directory structure so the c/escape symlink stays 1881 + + # inside the path 1882 + + arc.add("a/t/dummy") 1883 + + # Create b/ directory 1884 + + arc.add("b/") 1885 + + # Point "c" to the bottom of the tree in "a" 1886 + + arc.add("c", symlink_to=os.path.join("a", "t")) 1887 + + # link to non-existant location under "a" 1888 + + arc.add("c/escape", symlink_to=os.path.join("..", "..", 1889 + + "link_here")) 1890 + + # Move "c" to point to "b" ("c/escape" no longer exists) 1891 + + arc.add("c", symlink_to="b") 1892 + + # Attempt to create a hard link to "c/escape". Since it doesn't 1893 + + # exist it will attempt to extract "cescape" but at "boom". 1894 + + arc.add("boom", hardlink_to=os.path.join("c", "escape")) 1895 + + 1896 + + with self.check_context(arc.open(), 'data'): 1897 + + if not os_helper.can_symlink(): 1898 + + # When 'c/escape' is extracted, 'c' is a regular 1899 + + # directory, and 'c/escape' *would* point outside 1900 + + # the destination if symlinks were allowed. 1901 + + self.expect_exception( 1902 + + tarfile.LinkOutsideDestinationError) 1903 + + elif sys.platform == "win32": 1904 + + # On Windows, 'c/escape' points outside the destination 1905 + + self.expect_exception(tarfile.LinkOutsideDestinationError) 1906 + + else: 1907 + + e = self.expect_exception( 1908 + + tarfile.LinkFallbackError, 1909 + + "link 'boom' would be extracted as a copy of " 1910 + + + "'c/escape', which was rejected") 1911 + + self.assertIsInstance(e.__cause__, 1912 + + tarfile.LinkOutsideDestinationError) 1913 + + for filter in 'tar', 'fully_trusted': 1914 + + with self.subTest(filter), self.check_context(arc.open(), filter): 1915 + + if not os_helper.can_symlink(): 1916 + + self.expect_file("a/t/dummy") 1917 + + self.expect_file("b/") 1918 + + self.expect_file("c/") 1919 + + else: 1920 + + self.expect_file("a/t/dummy") 1921 + + self.expect_file("b/") 1922 + + self.expect_file("a/t/escape", symlink_to='../../link_here') 1923 + + self.expect_file("boom", symlink_to='../../link_here') 1924 + + self.expect_file("c", symlink_to='b') 1925 + + 1926 + + @symlink_test 1927 + + def test_exfiltration_via_symlink(self): 1928 + + # (CVE-2025-4138) 1929 + + # Test changing symlinks that result in a symlink pointing outside 1930 + + # the extraction directory, unless prevented by 'data' filter's 1931 + + # normalization. 1932 + + with ArchiveMaker() as arc: 1933 + + arc.add("escape", symlink_to=os.path.join('link', 'link', '..', '..', 'link-here')) 1934 + + arc.add("link", symlink_to='./') 1935 + + 1936 + + for filter in 'tar', 'data', 'fully_trusted': 1937 + + with self.check_context(arc.open(), filter): 1938 + + if os_helper.can_symlink(): 1939 + + self.expect_file("link", symlink_to='./') 1940 + + if filter == 'data': 1941 + + self.expect_file("escape", symlink_to='link-here') 1942 + + else: 1943 + + self.expect_file("escape", 1944 + + symlink_to='link/link/../../link-here') 1945 + + else: 1946 + + # Nothing is extracted. 1947 + + pass 1948 + + 1949 + + @symlink_test 1950 + + def test_chmod_outside_dir(self): 1951 + + # (CVE-2024-12718) 1952 + + # Test that members used for delayed updates of directory metadata 1953 + + # are (re-)filtered. 1954 + + with ArchiveMaker() as arc: 1955 + + # "pwn" is a veeeery innocent symlink: 1956 + + arc.add("a/pwn", symlink_to='.') 1957 + + # But now "pwn" is also a directory, so it's scheduled to have its 1958 + + # metadata updated later: 1959 + + arc.add("a/pwn/", mode='drwxrwxrwx') 1960 + + # Oops, "pwn" is not so innocent any more: 1961 + + arc.add("a/pwn", symlink_to='x/../') 1962 + + # Newly created symlink points to the dest dir, 1963 + + # so it's OK for the "data" filter. 1964 + + arc.add('a/x', symlink_to=('../')) 1965 + + # But now "pwn" points outside the dest dir 1966 + + 1967 + + for filter in 'tar', 'data', 'fully_trusted': 1968 + + with self.check_context(arc.open(), filter) as cc: 1969 + + if not os_helper.can_symlink(): 1970 + + self.expect_file("a/pwn/") 1971 + + elif filter == 'data': 1972 + + self.expect_file("a/x", symlink_to='../') 1973 + + self.expect_file("a/pwn", symlink_to='.') 1974 + + else: 1975 + + self.expect_file("a/x", symlink_to='../') 1976 + + self.expect_file("a/pwn", symlink_to='x/../') 1977 + + if sys.platform != "win32": 1978 + + st_mode = cc.outerdir.stat().st_mode 1979 + + self.assertNotEqual(st_mode & 0o777, 0o777) 1980 + + 1981 + + def test_link_fallback_normalizes(self): 1982 + + # Make sure hardlink fallbacks work for non-normalized paths for all 1983 + + # filters 1984 + + with ArchiveMaker() as arc: 1985 + + arc.add("dir/") 1986 + + arc.add("dir/../afile") 1987 + + arc.add("link1", hardlink_to='dir/../afile') 1988 + + arc.add("link2", hardlink_to='dir/../dir/../afile') 1989 + + 1990 + + for filter in 'tar', 'data', 'fully_trusted': 1991 + + with self.check_context(arc.open(), filter) as cc: 1992 + + self.expect_file("dir/") 1993 + + self.expect_file("afile") 1994 + + self.expect_file("link1") 1995 + + self.expect_file("link2") 1996 + + 1997 + def test_modes(self): 1998 + # Test how file modes are extracted 1999 + # (Note that the modes are ignored on platforms without working chmod) 2000 + @@ -4072,7 +4346,7 @@ def test_tar_filter(self): 2001 + # The 'tar' filter returns TarInfo objects with the same name/type. 2002 + # (It can also fail for particularly "evil" input, but we don't have 2003 + # that in the test archive.) 2004 + - with tarfile.TarFile.open(tarname) as tar: 2005 + + with tarfile.TarFile.open(tarname, encoding="iso8859-1") as tar: 2006 + for tarinfo in tar.getmembers(): 2007 + try: 2008 + filtered = tarfile.tar_filter(tarinfo, '') 2009 + @@ -4084,7 +4358,7 @@ def test_tar_filter(self): 2010 + def test_data_filter(self): 2011 + # The 'data' filter either raises, or returns TarInfo with the same 2012 + # name/type. 2013 + - with tarfile.TarFile.open(tarname) as tar: 2014 + + with tarfile.TarFile.open(tarname, encoding="iso8859-1") as tar: 2015 + for tarinfo in tar.getmembers(): 2016 + try: 2017 + filtered = tarfile.data_filter(tarinfo, '') 2018 + @@ -4242,13 +4516,13 @@ def valueerror_filter(tarinfo, path): 2019 + # If errorlevel is 0, errors affected by errorlevel are ignored 2020 + 2021 + with self.check_context(arc.open(errorlevel=0), extracterror_filter): 2022 + - self.expect_file('file') 2023 + + pass 2024 + 2025 + with self.check_context(arc.open(errorlevel=0), filtererror_filter): 2026 + - self.expect_file('file') 2027 + + pass 2028 + 2029 + with self.check_context(arc.open(errorlevel=0), oserror_filter): 2030 + - self.expect_file('file') 2031 + + pass 2032 + 2033 + with self.check_context(arc.open(errorlevel=0), tarerror_filter): 2034 + self.expect_exception(tarfile.TarError) 2035 + @@ -4259,7 +4533,7 @@ def valueerror_filter(tarinfo, path): 2036 + # If 1, all fatal errors are raised 2037 + 2038 + with self.check_context(arc.open(errorlevel=1), extracterror_filter): 2039 + - self.expect_file('file') 2040 + + pass 2041 + 2042 + with self.check_context(arc.open(errorlevel=1), filtererror_filter): 2043 + self.expect_exception(tarfile.FilterError) 2044 + diff --git a/Misc/NEWS.d/next/Security/2025-06-02-11-32-23.gh-issue-135034.RLGjbp.rst b/Misc/NEWS.d/next/Security/2025-06-02-11-32-23.gh-issue-135034.RLGjbp.rst 2045 + new file mode 100644 2046 + index 00000000000000..08a0087e203671 2047 + --- /dev/null 2048 + +++ b/Misc/NEWS.d/next/Security/2025-06-02-11-32-23.gh-issue-135034.RLGjbp.rst 2049 + @@ -0,0 +1,6 @@ 2050 + +Fixes multiple issues that allowed ``tarfile`` extraction filters 2051 + +(``filter="data"`` and ``filter="tar"``) to be bypassed using crafted 2052 + +symlinks and hard links. 2053 + + 2054 + +Addresses :cve:`2024-12718`, :cve:`2025-4138`, :cve:`2025-4330`, and :cve:`2025-4517`. 2055 + +
+3
pkgs/development/interpreters/python/cpython/default.nix
··· 340 ++ optionals (pythonAtLeast "3.13") [ 341 ./3.13/virtualenv-permissions.patch 342 ] 343 ++ optionals mimetypesSupport [ 344 # Make the mimetypes module refer to the right file 345 ./mimetypes.patch
··· 340 ++ optionals (pythonAtLeast "3.13") [ 341 ./3.13/virtualenv-permissions.patch 342 ] 343 + ++ optionals (pythonAtLeast "3.14") [ 344 + ./3.14/CVE-2025-4517.patch 345 + ] 346 ++ optionals mimetypesSupport [ 347 # Make the mimetypes module refer to the right file 348 ./mimetypes.patch