Fix kind load command for kfp image. #882
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
In recent transform refactoring PRs, such as those for ededup and tokenization, the
kind load
command is executed twice for the transform image in the kfp tests. This happens instead of executing it once for thekfp-data-processing
image and once for the transform image. This issue can be observed in the logs of KFP runs in these PRs by searching for thekind load
string.This PR aims to address this issue.
Explanation of the Error:
To understand why
kfp-data-processing:latest quay.io/dataprep1/data-prep-kit/kfp-data-processing:latest
is not loaded as expected in kfp-load-kind target let's look at the Makefile in kfp/kfp_ray_components:in its first line it includes
.make.defaults
which definesDOCKER_REMOTE_IMAGE
:This is a recursively expanded variable, meaning its value is dynamically recalculated each time it is referenced, using the current values of any variables it depends on.
The recent transform refactoring PRs define the
DOCKER_IMAGE_NAME
variable in kfp_ray/Makefile as follows:Which is also recursively expanded variable, causing
DOCKER_REMOTE_IMAGE
to be as follows:This explains why when executing
kind-load-image
targetDOCKER_REMOTE_IMAGE
equals toquay.io/dataprep1/data-prep-kit/ededup-ray:latest
.