Skip to content

Design: add a small karta.Workload interface - one clear front door for consumers #341

Description

@AviadHayumi

the feature

today a consumer wires karta from three packages : build a factory from pkg/resource , read through pkg/tree , mutate through component methods. it works , but the front door is not obvious - you have to know which package does what , and everything is concrete types.

i propose ONE small interface as the front door :

package karta

type Workload interface {
	Tree(ctx context.Context) (*tree.WorkloadTree, error)      // read everything : components , instances , status , scale
	UpdatePodTemplate(ctx context.Context, component string,
		update PodTemplateUpdate, opts ...UpdateOption) error  // mutate a component's pods
	Suspend(ctx context.Context) error
	Resume(ctx context.Context) error
	Components() []ComponentInfo                                // discover : writable pod fields , suspendable
	Object() (resource.KubernetesObject, error)                 // the mutated cr , ready to apply
}

func New(definition *v1alpha1.Karta, object resource.KubernetesObject) (Workload, error)

~6 methods , the verbs a consumer actually uses. our own AGENTS.md already says constructors return interface types when an interface exists - this makes that true at the top level.

one small interface , one clear front door

mutation is one verb with two doors. PodTemplateUpdate is a one-method interface and both patch types implement it , so the same verb accepts both ( like io.Copy accepting any reader ) :

type PodTemplateUpdate interface {
	AsPodMergePatch() Patch
}
// typed - compile safe , small set of known fields
w.UpdatePodTemplate(ctx, "worker", karta.PodPatch{
    SchedulerName: ptr.To("kai-scheduler"),
})

// raw - ANY field the definition can route , merge-patch style
w.UpdatePodTemplate(ctx, "worker", karta.Patch{
    "spec": map[string]any{
        "tolerations": []any{map[string]any{"key": "gpu", "operator": "Exists"}},
    },
})
  • the typed PodPatch compiles into the raw form , so ONE router serves both and the typed fields can never drift from what the engine routes
  • the raw door means we dont maintain an enum of pod fields forever : anything under the pod template routes if the definition has a writable path for it. the patch is validated against a partial corev1.PodTemplateSpec - k8s itself is the schema , a typo like spec.labels fails before anything runs
  • the merge rules : maps merge , scalars replace , lists are replaced as a whole - except spec.containers , which merges by container name. one unnamed containers entry means the sole container. null never deletes in v1
  • a field is writable only when its jq path can be assigned directly. formulas and computed projections return a typed unsupported error before any jq runs - they never get a corrupting write
  • mutations run on a copy , and we keep the copy only when everything worked - on any error the object is unchanged
  • unit tests get kartatest.Fake , which validates through the exported ValidatePodTemplateUpdate - the exact code path production runs , so the fake and production cannot disagree

UpdateOption is the standard functional options pattern ( like controller-runtime's client.InNamespace ). the patch says WHAT to change , options say how to apply it. today there is one option - WithInstances , for components that are really N instances in one cr ( lws workers for example ) :

w.UpdatePodTemplate(ctx, "worker", patch)                           // all instances
w.UpdatePodTemplate(ctx, "worker", patch, karta.WithInstances("b")) // only worker b

unknown ids fail before any write , and a future option ( dry-run for example ) is a new function - no breaking change , no new method on the interface.

real-life example : patch only the workers job of a jobset

a jobset is ONE cr holding several replicated jobs , each with its own pod template. in the catalog definition this is one karta component whose instances are the replicated jobs ( instanceIdPath: .spec.replicatedJobs[].name ) :

apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
  name: training
spec:
  replicatedJobs:
    - name: driver
      template:
        spec:
          template:
            spec:
              containers:
                - name: main
                  image: trainer:v1
    - name: workers
      replicas: 8
      template:
        spec:
          template:
            spec:
              containers:
                - name: main
                  image: trainer:v1

say the new image is only for the workers , the driver must stay on v1 :

w.UpdatePodTemplate(ctx, "replicated-job",
    karta.PodPatch{Image: ptr.To("trainer:v2")},
    karta.WithInstances("workers"))

after the call , only the workers template changed :

    - name: driver
      # ...
                  image: trainer:v1   # untouched
    - name: workers
      # ...
                  image: trainer:v2   # only this one

without the option the same call updates BOTH templates. and karta.WithInstances("typo") fails with "unknown instance id" before anything is written.

the plumbing ( the jq engine ) moves under internal/ , so the compiler makes the interface the only door. consumers get pkg/karta , pkg/resource as the power api , and validation - nothing else compiles.

the rationale

this is how go itself solves the same problem , and we would be aligned with all of it :

https://pkg.go.dev/io#Writer

  • io.Writer - a tiny consumer-side interface , many implementations. code accepts the interface and never cares which one it got

https://pkg.go.dev/database/sql

  • database/sql - one public facade for users , implementations plug in underneath. the user code never changes when the implementation does

https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/client#Client

  • controller-runtime client.Client - the exact pattern in our own ecosystem : consumers hold a small interface , the machinery lives behind internal packages , and tests use a fake client

what we get :

  1. consumption is one import and one constructor - easy to document , easy to teach
  2. unit tests get a fake Workload - no cluster , no fixtures
  3. implementations can evolve freely behind the interface - consumer code does not change
  4. with internal/ , hiding the plumbing is enforced by the compiler , not by convention

status

implemented in #345 , ready for review.

how we know it does not break anything : a test walks every definition in docs/catalog , applies every writable field to a synthetic object , and checks only the named fields changed. we also replayed real recorded cluster states before and after the change - reading gives the exact same tree , and mutations change zero fields they were not asked to change.

heads up , this is a public api change - anyone importing the lower packages directly is affected. wdyt ?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    Priority

    None yet

    Target date

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions