helm: add Kubeflow Pipelines chart

[danish9039]

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

Adds the Kubeflow Pipelines Helm chart under applications/pipeline/helm,
co-located with the Kustomize component it wraps.

Kustomize stays the source of truth. The synchronization script builds both
supported overlays and writes deterministic payloads under manifests/, which
small templates load with .Files.Get. Helm does not evaluate that content, so
the Argo expressions inside upstream manifests survive verbatim.

Two scenarios, matching the two installable variants:

scenario	Kustomize path
platform-database	applications/pipeline/overlays
platform-k8s-native	applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user-k8s-native

How it works

flowchart TD
    A["applications/pipeline/overlays"] --> C["scripts/generate-pipelines-helm-manifests.py"]
    B["upstream/env/cert-manager/<br/>platform-agnostic-multi-user-k8s-native"] --> C
    C --> D["validate identity,<br/>no duplicates"]
    D --> E["partition: common,<br/>database-only, k8s-native-only"]
    E --> F["CustomResourceDefinition?<br/>resource-policy keep"]
    F --> G["manifests/*.yaml<br/>six payloads, plain YAML"]
    G --> H["templates/*.yaml<br/>one 3-line switch per payload"]
    H --> I["rendered output equals kustomize build"]

Loading

Payloads are not templates

Kubeflow Pipelines manifests embed Argo expressions such as {{workflow.name}}.
While the payloads lived under templates/, Helm evaluated them, so every
delimiter had to be escaped on the way out and unescaped by the template engine
on the way back. A missed escape corrupts a manifest silently, and the committed
payload was not a faithful record of the render.

They now live in manifests/ and are read with .Files.Get, which Helm does not
evaluate. escape_helm_delimiters and the 13 escaped delimiters are gone.

One payload per template file. Two .Files.Get includes in a single template
file concatenate without a document separator, which merges the last resource of
one payload into the first of the next. This was caught by the Kustomize
comparison during development — a ClusterRole had acquired a
CustomResourceDefinition's spec, and the result was still valid YAML.

The {{- if }} conditions also moved out of the generated files into
templates/, so the generator no longer decides when a payload applies.

Release namespace corrected to kubeflow

This changes the documented installation command and deserves a maintainer's
opinion.

Every namespaced resource this chart renders declares namespace: kubeflow, but
the chart, its README, the comparison scenarios and the end-to-end install script
all used kubeflow-system as the release namespace. A release installed
elsewhere stores its metadata in one namespace while modifying another, and
helm uninstall then deletes resources it does not appear to own — the reasoning
the Dashboard chart README already documents.

templates/validate-namespace.yaml now enforces kubeflow, matching the other
wrapper charts. Reverting this is a one-line change if you disagree.

Tests

tests/test_pipelines_helm_workflows.py is removed. It asserted that a workflow
file contains particular strings, which mirrors configuration rather than
checking behaviour: it fails only when someone edits the workflow, and the fix is
to edit the test. Four of its five assertions describe things a failing workflow
run reports anyway.

The tests that do check behaviour ran in no workflow at all. They now run in
the comparison workflow's chart-behavior-tests job, which already has Helm and
Kustomize available, under the component-first file names (pipelines_helm_chart_test.py,
pipelines_helm_manifest_generator_test.py, pipelines_helm_install_helper_test.py).

Still to come, deliberately not here

The common partition stays for now. Dropping it is part of the shared-engine
work that has to wait until #3524 and #3525 merge, because both carry
scripts/helm_manifest_generator.py and three divergent copies would be worse
than one duplicate generator.

📦 Dependencies

Depends on #3524 for scripts/helm_manifest_generator.py, which this chart will
adopt once that merges. Until then it keeps its own generator.

The comparison harness (#3577)
is merged, and this chart registers through its own
applications/pipeline/helm/ci/comparison.yaml; no shared comparison file is
touched. The descriptor declares the sixteen retained CustomResourceDefinitions
the generator annotates with helm.sh/resource-policy: keep.

🐛 Related Issues

Part of the Project 5 Helm chart work.

Validation

python3 tests/run_helm_kustomize_comparison.py kubeflow-pipelines --all-scenarios
python3 tests/run_helm_kustomize_comparison.py all         # every component
python3 tests/pipelines_helm_chart_test.py
python3 tests/pipelines_helm_manifest_generator_test.py
python3 scripts/generate-pipelines-helm-manifests.py --repository-root .
git diff --quiet applications/pipeline/helm/manifests       # byte-identical
grep -rc '{{ "{{" }}' applications/pipeline/helm/manifests  # zero
helm template kfp applications/pipeline/helm -n wrong       # fails on the guard
helm lint applications/pipeline/helm --namespace kubeflow
shellcheck tests/pipelines_helm_install.sh scripts/synchronize-pipelines-manifests.sh
black --check tests/ scripts/ && git diff --check

Verified locally against Helm 4.1.0 and Kustomize v5.8.1.

✅ Contributor Checklist

• I have tested these changes with kustomize. See Installation Prerequisites.
• All commits are signed-off to satisfy the DCO check.
• I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

---

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-community-distribution.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kimwnasptd for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[danish9039]

/retest

[Copilot]

Pull request overview

Copilot reviewed 32 out of 34 changed files in this pull request and generated no new comments.

Suppressed comments (4)

.github/workflows/helm-kustomize-comparison.yml:175

• This step never executes the two newly added test modules, and no other GitHub Actions workflow invokes them. Consequently, their assertions can fail without affecting continuous integration. Run both modules here.

    - name: Test Kubeflow Pipelines Helm behavior
      run: |
        python tests/test_pipelines_helm_chart.py
        python -m unittest tests/test_generate_pipelines_helm_manifests.py

.github/workflows/helm-kustomize-comparison.yml:28

• The two newly added test modules are absent from this path filter, so a later pull request that changes only either module will not start this workflow. Add both paths.

This issue also appears on line 172 of the same file.

    - tests/test_pipelines_helm_chart.py
    - tests/test_generate_pipelines_helm_manifests.py

applications/pipeline/helm/README.md:90

• This command uses Helm's default release namespace, but templates/validate-namespace.yaml rejects every namespace except kubeflow. It therefore does not lint the chart under its supported configuration; add the required namespace, as done in applications/dashboard/helm/README.md:203.

helm lint applications/pipeline/helm

.github/workflows/helm-pipelines-test.yaml:11

• This job installs Istio from common/istio, OAuth2-Proxy from common/oauth2-proxy, and the Profile Controller from applications/dashboard/upstream/profile-controller, but none of those inputs is included in the filter. Changes to these installation dependencies can therefore break this scenario without running the workflow. Add their paths.

    - common/cert-manager/**
    - common/kubeflow-namespace/**
    - common/kubeflow-roles/**