DNM checking jobs on IBM private cloud

[danpawlik]

No description provided.

[softwarefactory-project-zuul]

Zuul encountered a syntax error while parsing its
configuration in the repo infrawatch/service-telemetry-operator on branch stable-1.5. The
problem was:

Nodeset stf-crc_extracted-ocp414 does not match existing definition in branch master

The problem appears in the the "stf-crc_extracted-ocp414" nodeset stanza:

nodeset:
name: stf-crc_extracted-ocp414
nodes:
- name: controller
label: cloud-centos-9-stream-tripleo-vexxhost
- name: crc
label: coreos-crc-extracted-2-30-0-xxl

in "infrawatch/service-telemetry-operator/[email protected]", line 2

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/1e34ac5959bb49ee8e80fd09b8899b86

❌ stf-crc-ocp_414-local_build RETRY_LIMIT in 2m 37s
❌ stf-crc-ocp_416-local_build RETRY_LIMIT in 2m 37s
❌ stf-crc-ocp_414-local_build-index_deploy RETRY_LIMIT in 2m 34s
❌ stf-crc-ocp_416-local_build-index_deploy RETRY_LIMIT in 2m 35s
❌ stf-crc-ocp_414-nightly_bundles-index_deploy RETRY_LIMIT in 2m 36s
❌ stf-crc-ocp_416-nightly_bundles-index_deploy RETRY_LIMIT in 2m 35s

[danpawlik]

Error:

2025-02-07 07:59:35.159039 | TASK [Create the default network]
2025-02-07 02:59:36.330840 | controller | Error while executing command: BadRequestException: 400, Invalid input for operation: Requested MTU is too big, maximum is 1442.
2025-02-07 08:00:37.630141 | controller | ERROR
2025-02-07 08:00:37.630279 | controller | {
2025-02-07 08:00:37.630309 | controller |   "attempts": 10,
2025-02-07 08:00:37.630330 | controller |   "delta": "0:00:00.947745",
2025-02-07 08:00:37.630348 | controller |   "end": "2025-02-07 03:00:37.599675",
2025-02-07 08:00:37.630364 | controller |   "msg": "non-zero return code",
2025-02-07 08:00:37.630380 | controller |   "rc": 1,
2025-02-07 08:00:37.630397 | controller |   "start": "2025-02-07 03:00:36.651930"
2025-02-07 08:00:37.630413 | controller | }
failure

[danpawlik]

The correct way to pass on IBM is to use such config: https://github.com/openstack-k8s-operators/ci-framework/blob/main/zuul.d/edpm_multinode.yaml#L14

[danpawlik]

now there is an error with one pod:

openshift-kube-apiserver-operator                  kube-apiserver-operator-5c5bbcd9d4-2bz65                  1/1     Running     2             443d
openshift-kube-apiserver                           installer-11-crc-pjmnl-master-0                           0/1     Error       0             21h
openshift-kube-apiserver                           kube-apiserver-crc-pjmnl-master-0                         5/5     Running     10            21h

where:

  Normal   Created     3m48s  kubelet  Created container kube-apiserver-check-endpoints
  Normal   Started     3m48s  kubelet  Started container kube-apiserver-check-endpoints
  Warning  ProbeError  3m38s  kubelet  Readiness probe error: Get "https://192.168.126.11:17697/healthz": dial tcp 192.168.126.11:17697: connect: connection refused
body:
  Warning  Unhealthy   3m38s                  kubelet  Readiness probe failed: Get "https://192.168.126.11:17697/healthz": dial tcp 192.168.126.11:17697: connect: connection refused
  Warning  ProbeError  3m38s (x2 over 3m38s)  kubelet  Startup probe error: HTTP probe failed with statuscode: 403
body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/healthz\"","reason":"Forbidden","details":{},"code":403}
  Warning  Unhealthy  3m38s (x2 over 3m38s)  kubelet  Startup probe failed: HTTP probe failed with statuscode: 403

so there is a security group rules issue or something else

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/da48069d312b4a31a280a87caec2c3f2

❌ stf-crc-ocp_414-local_build RETRY_LIMIT in 19m 17s
❌ stf-crc-ocp_416-local_build RETRY_LIMIT in 4m 22s
❌ stf-crc-ocp_414-local_build-index_deploy RETRY_LIMIT in 19m 16s
❌ stf-crc-ocp_416-local_build-index_deploy RETRY_LIMIT in 4m 23s
❌ stf-crc-ocp_414-nightly_bundles-index_deploy RETRY_LIMIT in 18m 59s
❌ stf-crc-ocp_416-nightly_bundles-index_deploy RETRY_LIMIT in 5m 20s

[danpawlik]

recheck

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/afba6caa0ec842e5a9b323e90deef0ef

❌ stf-crc-ocp_414-local_build FAILURE in 18m 08s
✔️ stf-crc-ocp_416-local_build SUCCESS in 28m 07s
❌ stf-crc-ocp_414-local_build-index_deploy FAILURE in 18m 46s
✔️ stf-crc-ocp_416-local_build-index_deploy SUCCESS in 28m 34s
❌ stf-crc-ocp_414-nightly_bundles-index_deploy FAILURE in 18m 46s
✔️ stf-crc-ocp_416-nightly_bundles-index_deploy SUCCESS in 22m 10s