Skip to content

feat(app): replace Knative Serving with native Kubernetes Deployment and Service controller - #8228

Closed
chansoo-calico wants to merge 2 commits into
flyteorg:mainfrom
chansoo-calico:feat/native-k8s-app-controller
Closed

chansoo-calico wants to merge 2 commits into
flyteorg:mainfrom
chansoo-calico:feat/native-k8s-app-controller

Conversation

@chansoo-calico

Copy link
Copy Markdown

Why are the changes needed?

app/internal/k8s previously required Knative Serving (serving.knative.dev/v1 Service) and a Knative ingress controller (Istio/Kourier) to deploy and manage AppService / InternalAppService (app_env) workloads. On standard Kubernetes clusters (such as GKE without Knative installed), AppService could not start or deploy apps because the serving.knative.dev/v1 CRDs and controller were absent.

What changes were proposed in this pull request?

  • Replace serving.knative.dev/v1 Service management in app/internal/k8s, flytestdlib/app/app_scheme.go, and manager/cmd/components.go with native Kubernetes apps/v1 Deployment and core/v1 Service (ClusterIP) resources.
  • Propagate K8sPod metadata labels and annotations onto the Deployment pod template (filtering out job-only kueue.x-k8s.io/queue-name labels) and stamp flyte.org/managed=true and flyte.org/project on app replica pods for flyte-task-pod-identity admission policies and the executor pod cache.
  • Substitute {{.rawOutputDataPrefix}} in container Command and Args for fserve app containers.
  • Support path-based BaseDomain URLs (when BaseDomain contains /, e.g. https://<host>/apps/<service-name>/) in PublicIngress in addition to subdomain-based URLs.
  • Update app/internal/k8s/app_client_test.go to test Deployment + Service lifecycle, pod template metadata propagation, {{.rawOutputDataPrefix}} substitution, and path-based PublicIngress.

How was this patch tested?

Ran unit tests and built ./manager/cmd/:

go test -v ./app/... ./flytestdlib/app/...
go build ./manager/cmd/

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

…and Service controller

Replaces the Knative Serving (serving.knative.dev/v1 Service) dependency in app/internal/k8s with native Kubernetes apps/v1 Deployment and core/v1 ClusterIP Service resources so AppService / InternalAppService (app_env) runs on standard Kubernetes clusters without Knative or Istio/Kourier installed.

- Manages apps/v1 Deployment and core/v1 ClusterIP Service in AppK8sClient (Deploy, Stop, Delete, GetApp, List, GetReplicas, DeleteReplica, and Deployment informer watch).
- Preserves K8sPod metadata labels and annotations on the Deployment pod template (excluding Kueue job queue labels) and stamps flyte.org/managed and flyte.org/project on app replica pods so admission policies and the executor pod cache recognize them.
- Substitutes {{.rawOutputDataPrefix}} in fserve container command/args.
- Supports path-based BaseDomain URLs (e.g. https://<host>/apps/<service-name>/) in PublicIngress alongside subdomain routing.

Signed-off-by: Chansoo Lee <chansoo@calicolabs.com>
Copilot AI balanced review requested due to automatic review settings October 22, 2026 17:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Missing native-resource permissions and public routing block standard installations, while serving and rendering regressions remain unresolved.

9 open issues
What changed in this PR

Replaces the app controller’s Knative Serving resources with native Kubernetes Deployments and Services so app workloads can run on clusters without Knative.

Changes:

  • Migrates resource lifecycle, watches, status mapping, and scheme registration to native Kubernetes types.
  • Propagates pod metadata, renders raw-output placeholders, and adds path-based public URL formatting.
  • Updates lifecycle tests and adds metadata, placeholder, and URL coverage.
File Description
manager/​cmd/​components.go Registers Deployment types during manager setup.
flytestdlib/​app/​app_scheme.go Registers native app and core Kubernetes types.
app/​internal/​k8s/​app_client.go Implements Deployment/Service management and updated pod and URL handling.
app/​internal/​k8s/​app_client_test.go Adapts tests to native resources and covers new behavior.

Review effort: Balanced

Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go Outdated
Comment thread app/internal/k8s/app_client.go Outdated
Copilot AI balanced review requested due to automatic review settings October 22, 2026 19:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
@chansoo-calico
chansoo-calico force-pushed the feat/native-k8s-app-controller branch from 83e08dd to a5aab51 Compare October 22, 2026 19:47
Copilot AI balanced review requested due to automatic review settings October 22, 2026 22:00
@chansoo-calico
chansoo-calico force-pushed the feat/native-k8s-app-controller branch from a5aab51 to ea2596e Compare October 22, 2026 22:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
Copilot AI balanced review requested due to automatic review settings October 22, 2026 22:17
@chansoo-calico
chansoo-calico force-pushed the feat/native-k8s-app-controller branch from ea2596e to 54d2d5d Compare October 22, 2026 22:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Port-reference rewriting can redirect health checks, and stopped-state updates can remove the marker needed for accurate status reporting.

2 open issues
3 resolved since last review

Review effort: Balanced

Comment thread app/internal/k8s/app_client.go
Comment thread app/internal/k8s/app_client.go
…controller

Signed-off-by: Chansoo Lee <chansoo@calicolabs.com>
Copilot AI balanced review requested due to automatic review settings October 22, 2026 22:52
@chansoo-calico
chansoo-calico force-pushed the feat/native-k8s-app-controller branch from 54d2d5d to 9cc3631 Compare October 22, 2026 22:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Rollout status can report ACTIVE while only old replicas are ready and replacement replicas remain unavailable.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@popojk popojk closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants