Skip to content

Conversation

@neisw
Copy link

@neisw neisw commented Dec 11, 2025

Reverts #253

Per OpenShift policy, we are reverting this breaking change to get CI and/or nightly payloads flowing again.

Multiple payload failures due to console-operator panic

To unrevert this, revert this PR, and layer an additional separate commit on top that addresses the problem. Before merging the unrevert, please run these jobs on the PR and check the result of these jobs to confirm the fix has corrected the problem:

/payload-job periodic-ci-openshift-release-master-ci-4.21-e2e-azure-ovn-upgrade
/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

CC: @mdbooth

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Dec 11, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 11, 2025

@neisw: This pull request references TRT-2473 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.21.0" version, but no target version was set.

Details

In response to this:

Reverts #253

Per OpenShift policy, we are reverting this breaking change to get CI and/or nightly payloads flowing again.

Multiple payload failures due to console-operator panic

To unrevert this, revert this PR, and layer an additional separate commit on top that addresses the problem. Before merging the unrevert, please run these jobs on the PR and check the result of these jobs to confirm the fix has corrected the problem:

/payload-job periodic-ci-openshift-release-master-ci-4.21-e2e-azure-ovn-upgrade
/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

CC: @user

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 11, 2025

@neisw: This pull request references TRT-2473 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.21.0" version, but no target version was set.

Details

In response to this:

Reverts #253

Per OpenShift policy, we are reverting this breaking change to get CI and/or nightly payloads flowing again.

Multiple payload failures due to console-operator panic

To unrevert this, revert this PR, and layer an additional separate commit on top that addresses the problem. Before merging the unrevert, please run these jobs on the PR and check the result of these jobs to confirm the fix has corrected the problem:

/payload-job periodic-ci-openshift-release-master-ci-4.21-e2e-azure-ovn-upgrade
/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

CC: @mdbooth

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from nrb and theobarberbany December 11, 2025 01:49
@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: neisw
Once this PR has been reviewed and has the lgtm label, please assign chrischdi for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@neisw
Copy link
Author

neisw commented Dec 11, 2025

/payload-job periodic-ci-openshift-release-master-ci-4.21-e2e-azure-ovn-upgrade
/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@neisw: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b505fcb0-d633-11f0-9515-060a5767e64c-0

@neisw
Copy link
Author

neisw commented Dec 11, 2025

/pipeline-required

@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@neisw: This PR was included in a payload test run from openshift/api#2617
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/d46da960-d636-11f0-8a55-939474e54294-0

@wangke19
Copy link

/lgtm
Blocking CI jobs ran passed.

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 11, 2025
@wangke19
Copy link

/test e2e-vsphere-staticip-ipam-ovn

@neisw
Copy link
Author

neisw commented Dec 11, 2025

/override ci/prow/e2e-vsphere-staticip-ipam-ovn

New job for this attempted fix that we expect to be broken with the revert

@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@neisw: Overrode contexts on behalf of neisw: ci/prow/e2e-vsphere-staticip-ipam-ovn

Details

In response to this:

/override ci/prow/e2e-vsphere-staticip-ipam-ovn

New job for this attempted fix that we expect to be broken with the revert

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@neisw: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@neisw
Copy link
Author

neisw commented Dec 11, 2025

label /approved
/verified by payload-job

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Dec 11, 2025
@openshift-ci-robot
Copy link

@neisw: This PR has been marked as verified by payload-job.

Details

In response to this:

label /approved
/verified by payload-job

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@neisw
Copy link
Author

neisw commented Dec 11, 2025

/label approved

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 11, 2025
@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@neisw: This PR was included in a payload test run from openshift/origin#30586
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/aa700710-d67d-11f0-9eb6-b58676753e73-0

@damdo
Copy link
Member

damdo commented Dec 11, 2025

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 11, 2025
@damdo
Copy link
Member

damdo commented Dec 11, 2025

We are already investigating this in another thread.
Please give us a moment to figure out what's going on here if possible

@mdbooth
Copy link

mdbooth commented Dec 11, 2025

Reverting this would be a mistake. We should not revert this. We already understand why this breaks. If we revert it, we will still have to re-apply the same fix, and we will still have to address this issue. We should address it now, instead.

I will write up a detailed timeline with all the details shortly.

@neisw
Copy link
Author

neisw commented Dec 11, 2025

/close

After reviewing timelines on when the root issue came in: #243 12/3 and the fact it was not yet in a released EC we manually accepted payloads to get micro upgrades unstuck

@openshift-ci openshift-ci bot closed this Dec 11, 2025
@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@neisw: Closed this PR.

Details

In response to this:

/close

After reviewing timelines on when the root issue came in: #243 12/3 and the fact it was not yet in a released EC we manually accepted payloads to get micro upgrades unstuck

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@neisw: This PR was included in a payload test run from openshift/cluster-kube-controller-manager-operator#900
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/3ba3f640-d6f8-11f0-9224-5e46dbc55cc5-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@neisw: This PR was included in a payload test run from openshift/machine-config-operator#5489
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/03eb04e0-d6f9-11f0-8261-7255b3fcf62c-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@neisw: This PR was included in a payload test run from openshift/origin#30604
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/df1743d0-d6f9-11f0-82e3-520759d1a342-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@bertinatto: This PR was included in a payload test run from openshift/cluster-kube-controller-manager-operator#900
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/dab13350-d735-11f0-8fed-fcd0fe4ceb51-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@neisw: This PR was included in a payload test run from openshift/machine-config-operator#5489
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b521a1b0-d749-11f0-9c6f-49bba9ef02f5-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@ngopalak-redhat: This PR was included in a payload test run from openshift/machine-config-operator#5491
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/528debb0-d755-11f0-826b-d2c4b80a4f0a-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@neisw: This PR was included in a payload test run from openshift/machine-config-operator#5489
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/7c333c90-d755-11f0-917b-f286c81cb85b-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 12, 2025

@neisw: This PR was included in a payload test run from openshift/machine-config-operator#5489
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.22-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/80db55f0-d757-11f0-9eab-e3cee46cb112-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 16, 2025

@ngopalak-redhat: This PR was included in a payload test run from openshift/machine-config-operator#5491
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a07ff0e0-da55-11f0-885e-d2dbc5e46d89-0

@openshift-ci
Copy link

openshift-ci bot commented Dec 17, 2025

@ngopalak-redhat: This PR was included in a payload test run from openshift/machine-config-operator#5491
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/953e4e20-daf0-11f0-88c3-4525b8822acb-0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants