tools/wpt tests failing on Python 3.12.8

[jonathan-j-lee]

Examples across several PRs:

• https://github.com/web-platform-tests/wpt/pull/49520/checks?check_run_id=33919806185
• https://github.com/web-platform-tests/wpt/pull/49529/checks?check_run_id=33972329964
• https://github.com/web-platform-tests/wpt/pull/49513/checks?check_run_id=33960289324

Sample failure:

self = PosixPath('/home/test/web-platform-tests/tools/wpt/.tox/py312/lib/python3.12/site-packages')

other = PosixPath('/home/test/web-platform-tests/_venv3'), walk_up = False

_deprecated = (), step = 0



    def relative_to(self, other, /, *_deprecated, walk_up=False):

        """Return the relative path to another path identified by the passed

        arguments.  If the operation is not possible (because this is not

        related to the other path), raise ValueError.

    

        The *walk_up* parameter controls whether `..` may be used to resolve

        the path.

        """

        if _deprecated:

            msg = ("support for supplying more than one positional argument "

                   "to pathlib.PurePath.relative_to() is deprecated and "

                   "scheduled for removal in Python {remove}")

            warnings._deprecated("pathlib.PurePath.relative_to(*args)", msg,

                                 remove=(3, 14))

        other = self.with_segments(other, *_deprecated)

        for step, path in enumerate([other] + list(other.parents)):

            if self.is_relative_to(path):

                break

            elif not walk_up:

>               raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}")

E               ValueError: '/home/test/web-platform-tests/tools/wpt/.tox/py312/lib/python3.12/site-packages' is not in the subpath of '/home/test/web-platform-tests/_venv3'

The theory in the WPT matrix chat is nested virtualenvs, but it doesn't look like the recent commit log for tools/ contains any likely culprits.

[gsnedders]

https://community-tc.services.mozilla.com/tasks/IaWJRR5pQ0OLBXsD4dL_jw/runs/0/logs/public/logs/live.log is one of the last successful runs,

Successfully installed cachetools-5.5.0 chardet-5.2.0 colorama-0.4.6 distlib-0.3.9 filelock-3.16.1 packaging-24.2 platformdirs-4.3.6 pluggy-1.5.0 pyproject-api-1.8.0 tomli-2.2.1 tox-4.23.2 typing-extensions-4.12.2 virtualenv-20.28.0

Whereas https://community-tc.services.mozilla.com/tasks/JN89dLVhRkmS80b45O_CPQ/runs/0/logs/public/logs/live.log:

Successfully installed cachetools-5.5.0 chardet-5.2.0 colorama-0.4.6 distlib-0.3.9 filelock-3.16.1 packaging-24.2 platformdirs-4.3.6 pluggy-1.5.0 pyproject-api-1.8.0 tomli-2.2.1 tox-4.23.2 typing-extensions-4.12.2 virtualenv-20.28.0

So it's not even like tox or one of its dependencies changed version.

The former was based on master 4817834, the latter based on 1d55057, and there are no changes in tools between them.

We did however change Python version:

 Selecting previously unselected package python3.12.
-Preparing to unpack .../python3.12_3.12.7-1+focal1_amd64.deb ...
-Unpacking python3.12 (3.12.7-1+focal1) ...
+Preparing to unpack .../python3.12_3.12.8-1+focal1_amd64.deb ...
+Unpacking python3.12 (3.12.8-1+focal1) ...

Which thus makes that the most suspicious part; see https://docs.python.org/release/3.12.8/whatsnew/changelog.html#python-3-12-8-final for the full changelog.

My first guess is:

gh-126789: Fixed the values of sysconfig.get_config_vars(), sysconfig.get_paths(), and their siblings when the site initialization happens after sysconfig has built a cache for sysconfig.get_config_vars().

In which case it is likely our self-activating virtualenv script at fault.

[gsnedders]

(I don't actually have time to particularly investigate this further right now, but I strongly suspect it's that test environment configuration change. This is what we get for non-reproducible environments. 😐)

[WeizhongX]

Hit this again in #49605.

Also when I tried to rerun this task by clicking rerun, I got the below:

Uh oh! Looks like an error!
Client ID static/taskcluster/github does not have sufficient scopes and is missing the following scopes:

{
  "AnyOf": [
    "queue:rerun-task:taskcluster-github/ZlESXRiRRQmMCspCfS96Jw/G-tz052ERjOnqHDu1Wb3WA",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:taskcluster-github/ZlESXRiRRQmMCspCfS96Jw"
      ]
    }
  ]
}
This request requires the client to satisfy the following scope expression:

{
  "AnyOf": [
    "queue:rerun-task:taskcluster-github/ZlESXRiRRQmMCspCfS96Jw/G-tz052ERjOnqHDu1Wb3WA",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:taskcluster-github/ZlESXRiRRQmMCspCfS96Jw"
      ]
    }
  ]
}

This forces us to use close and reopen which will rerun all the tasks.

[jonathan-j-lee]

This forces us to use close and reopen which will rerun all the tasks.

Apparently not easily fixable: #39458