Skip to content

Ensure upgrade tests do not test past the specified version#10112

Draft
desrosj wants to merge 49 commits intoWordPress:trunkfrom
desrosj:try/upgrade-matrix-generation
Draft

Ensure upgrade tests do not test past the specified version#10112
desrosj wants to merge 49 commits intoWordPress:trunkfrom
desrosj:try/upgrade-matrix-generation

Conversation

@desrosj
Copy link
Copy Markdown
Member

@desrosj desrosj commented Oct 1, 2025

Supersedes #10099.

Currently, if you manually dispatch a run of the upgrade testing workflow with a version that is lower than the currently supported major version, the tests run past the specified version instead of stopping when appropriate.

For example, this run was dispatched for version 6.6.4. The jobs spawned included "Upgrading 6.7 to 6.6.4" and "Upgrading 6.8 to 6.6.4)" despite 6.6.4 being older than those two major versions.

Screenshot 2025-10-01 at 9 46 40 PM

The logic does not account for this and no upgrade is actually performed (technically this is a scenario where a downgrade would be tested) because the --force argument is not passed to the wp core update command.

Screenshot 2025-10-01 at 9 47 17 PM

This PR adds consideration for this scenario through a new job that generates a list of exclusions for all following strategy matrices.

Trac ticket:


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

@desrosj desrosj self-assigned this Oct 1, 2025
@desrosj desrosj changed the title Upgrade tests should not run past the new version (option 2) Ensure upgrade tests do not test past the specified version Oct 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant