Remove digest for pushing to local registry but keep it when pulling

[mgencur]

When there are additional assets to be installed during OpenShift installation and they point to images by digests, the installation fails as it can't push images by digest. See the error below that was caught in this run

 +(utils.sh:158): create_cluster(): sudo podman push --tls-verify=false --remove-signatures --authfile /opt/dev-scripts/pull_secret.json registry.connect.redhat.com/isovalent/cilium-olm@sha256:9ab6be29447125e886300e9258b9a06bedf0a9d87405832aa8b6565ed1ba4215 virthost.ostest.test.metalkube.org:5000/localimages/assets/cilium-olm@sha256:9ab6be29447125e886300e9258b9a06bedf0a9d87405832aa8b6565ed1ba4215
Error: Copying this image would require changing layer representation, which we cannot do: "Destination specifies a digest" 

The fix does not use the digest for pushing. It pushes without the digest and then gets the target image digest by podman inspect and uses it in the final sed replacement in manifests.

Tried this fix manually step by step. Results below.

sh-5.1# IMAGE=quay.io/cilium/cilium@sha256:351d6685dc6f6ffbcd5451043167cfa8842c6decf80d8c8e426a417c73fb56d4

sh-5.1# IMAGE_SHORT=${IMAGE##*/}

sh-5.1# echo $IMAGE_SHORT
cilium@sha256:351d6685dc6f6ffbcd5451043167cfa8842c6decf80d8c8e426a417c73fb56d4

sh-5.1# [[ $IMAGE_SHORT =~ "@" ]] && DIGEST=${IMAGE_SHORT##*@}

sh-5.1# echo $DIGEST
sha256:351d6685dc6f6ffbcd5451043167cfa8842c6decf80d8c8e426a417c73fb56d4

sh-5.1# IMAGE_SHORT=${IMAGE_SHORT%@*}

sh-5.1# echo $IMAGE_SHORT
cilium

sh-5.1# IMAGE_MIRRORED=image-registry.openshift-image-registry.svc:5000/default/${IMAGE_SHORT}

sh-5.1# echo $IMAGE_MIRRORED
image-registry.openshift-image-registry.svc:5000/default/cilium

sh-5.1# podman pull $IMAGE
Trying to pull quay.io/cilium/cilium@sha256:351d6685dc6f6ffbcd5451043167cfa8842c6decf80d8c8e426a417c73fb56d4...
Getting image source signatures
Copying blob dfc06430d5d3 skipped: already exists  
Copying blob ea5b2034a45c skipped: already exists  
Copying blob 80a2e9137f74 skipped: already exists  
Copying blob 7f7601a46f5f skipped: already exists  
Copying blob b25c4ee33d80 skipped: already exists  
Copying blob 9ab57caf9562 skipped: already exists  
Copying blob d52d6fccae91 skipped: already exists  
Copying blob a0076705e219 skipped: already exists  
Copying config 9d2c079a4c done   | 
Writing manifest to image destination
9d2c079a4c49baf3a409411cfd90f2a524892b588429719f9c366582f38e3865

sh-5.1# podman push --remove-signatures $IMAGE $IMAGE_MIRRORED
Copying blob 423b6553e41e skipped: already exists  
Copying blob b3da73419f9a skipped: already exists  
Copying blob f24cc031c098 skipped: already exists  
Copying blob 15be1e6357f2 skipped: already exists  
Copying blob 3db070c450f8 skipped: already exists  
Copying blob fb82c0c0f0c0 skipped: already exists  
Copying blob 7d9ac9e1aa36 skipped: already exists  
Copying blob a55251097512 skipped: already exists  
Copying config 9d2c079a4c done   | 
Writing manifest to image destination

sh-5.1# DIGEST=$(podman inspect --format "{{.Digest}}" $IMAGE_MIRRORED)

sh-5.1# echo $DIGEST
sha256:0512f77f1acc748b053972b70e076a8a6bd815e5c35bfb25f1ae591b745ddf82

sh-5.1# IMAGE_MIRRORED="${IMAGE_MIRRORED}@${DIGEST}"
sh-5.1# podman pull $IMAGE_MIRRORED
Trying to pull image-registry.openshift-image-registry.svc:5000/default/cilium@sha256:0512f77f1acc748b053972b70e076a8a6bd815e5c35bfb25f1ae591b745ddf82...
Getting image source signatures
Copying blob b3da73419f9a skipped: already exists  
Copying blob 15be1e6357f2 skipped: already exists  
Copying blob f24cc031c098 skipped: already exists  
Copying blob 3db070c450f8 skipped: already exists  
Copying blob 423b6553e41e skipped: already exists  
Copying blob 7d9ac9e1aa36 skipped: already exists  
Copying blob a55251097512 skipped: already exists  
Copying blob fb82c0c0f0c0 skipped: already exists  
Copying config 9d2c079a4c done   | 
Writing manifest to image destination
9d2c079a4c49baf3a409411cfd90f2a524892b588429719f9c366582f38e3865

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign andfasano for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @mgencur. Thanks for your PR.

I'm waiting for a openshift-metal3 member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[celebdor]

/ok-to-test

[openshift-ci]

@mgencur: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-agent-sno-ipv6	4f7958f	link	false	/test e2e-agent-sno-ipv6
ci/prow/e2e-metal-ipi-virtualmedia	4f7958f	link	false	/test e2e-metal-ipi-virtualmedia
ci/prow/e2e-metal-ipi-bm-bond	4f7958f	link	false	/test e2e-metal-ipi-bm-bond
ci/prow/e2e-metal-ipi-serial-ovn-ipv6	4f7958f	link	false	/test e2e-metal-ipi-serial-ovn-ipv6
ci/prow/e2e-metal-ipi-ovn-ipv6	4f7958f	link	true	/test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-metal-ipi-bm	4f7958f	link	true	/test e2e-metal-ipi-bm
ci/prow/e2e-metal-ipi-ovn-dualstack	4f7958f	link	false	/test e2e-metal-ipi-ovn-dualstack
ci/prow/e2e-metal-ipi-serial-ipv4	4f7958f	link	true	/test e2e-metal-ipi-serial-ipv4

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.