Apply tolerations to ray pods defined by env var in KFP v2

[revit13]

Search before asking

• I searched the issues and found no similar issues.

Component

KFP workflows

Feature

Related to #620 and PRs #643 and #627

Following @roytman request in this pr tolerations for ray pods (and kfp pods) can be applied for KFP v1 when exporting the following env var before compiling the pipeline; however, for KFPv2 its is not supported yet.

export KFP_TOLERATIONS='[{"key": "key","operator": "Equal", "value1": "value", "effect": "NoSchedule"}]'

In addition, for both KFP v1 and v2 the tolerations for ray pods can be specified in the pipeline params: ray_head_options and ray_worker_options.
for example:

  "cpu": 1,
  "image": "quay.io/dataprep1/data-prep-kit/noop-ray:latest",
  "memory": 4,
  "tolerations": [
    {
      "key": "preprocessing",
      "operator": "Equal",
      "value": "dedicated",
      "effect": "NoSchedule"
    }
  ]
}

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[roytman]

Done with #627 and #643