Repository navigation
IDL: action input and code overrides and a fork marker for recovery runs - #8234
cosmicbboy wants to merge 6 commits into
Conversation
Recover gains action_input_overrides (action names + data-plane URI of ActionInputOverridesData) and server-set implied_rerun_actions. The actions Action and ActionMetadata carry input_override_applied, RunDetails reports per-override status, and CreateRunResponse carries warnings. Signed-off-by: Niels Bantilan <niels@union.ai> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fork that only changes code carries no action input overrides, so clients could not tell it apart from a plain recover. Recover.fork records that the run is a fork; it is informational and does not change recovery. Also regenerates the Rust bindings, which missed the earlier action-input-overrides fields. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Add ActionInputOverridesData.code_bundles: action name -> ActionCodeBundle, the code bundle archive that one action of a fork runs in place of the one its parent passes down. The parent's controller applies it when it enqueues the action, like the action's input overrides, so a fork can patch the code of a single task action without renaming it or changing the rest of the run. The bundle URI lives in the overrides file in the data plane; the RunSpec still carries only action names and the file's location. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
cosmicbboy
changed the title
…nput-overrides # Conflicts: # gen/go/flyteidl2/actions/actions_service.pb.go # gen/python/flyteidl2/actions/actions_service_pb2.py # gen/rust/src/flyteidl2.actions.rs # gen/rust/src/flyteidl2.workflow.rs # gen/ts/flyteidl2/actions/actions_service_pb.ts
Review feedback: ActionInputOverridesData now maps each overridden action to the URI of a serialized Inputs in the data plane (input_uris), instead of embedding the Inputs. This matches how an action's inputs are referenced (input_uri), keeps the overrides file small however many overrides a fork carries, and lets the parent's controller read only the override inputs of the actions it launches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Only generated bindings conflicted; the .proto sources merged cleanly (main's queue-scoped ListActions, ActionStatus queue timestamps and TaskActionMetadata.queue sit beside the override fields). Regenerated gen/ with buf 1.58.0, the version gen.Dockerfile pins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
What
IDL for forking a run: a recovery run (
RELATION_TYPE_RECOVER) that reuses the source run's succeeded actions but changes the inputs and/or the code of specific task actions, or the code of the whole run.task.Recoveraction_input_overrides(ActionInputOverrides): names of the overridden task actions, plus the data-plane URI of a serializedActionInputOverridesData. Override values and code bundle locations never travel in theRunSpec.implied_rerun_actions: set by the server, the ancestors that must re-execute so an overridden action is enqueued again.fork: marks a recovery run as a fork, so a fork that only changes code (no overrides) can be told apart from a plain recover. Informational; it does not change recovery.task.ActionInputOverridesDatainput_uris: map of action name to the data-plane URI of a serializedInputsholding the inputs that change (like an action'sinput_uri). The overrides file stays small however many overrides a fork carries, and a parent's controller reads only the override inputs of the actions it launches.code_bundles: map of action name toActionCodeBundle(archive URI and version). The parent's controller runs that one action from this bundle instead of the one it passes down, so a fork can patch a single task action's code without renaming it or changing the rest of the run.actions.Action.input_override_applied/workflow.ActionMetadata.input_override_applied: set by the parent's controller when it applied an override (inputs or code).workflow.RunDetails.action_input_override_status: per-override status (PENDING/APPLIED/UNAPPLIED), derived on read.workflow.CreateRunResponse.warnings: non-fatal warnings (e.g. an override nested under another override).All fields are new and optional; existing clients and servers are unaffected, and a backend that does not implement forks ignores them.
Generated code
Regenerated Go, Python, TypeScript, and Rust bindings.
Note: the bindings were generated with
buf1.50.0 (the repo's--cleanflag needs a newerbufthan some local installs have). It would be worth re-runningmake genin CI to confirm the output matches the pinned toolchain.Testing
go build ./gen/go/...buf lint(clean apart from the pre-existingflytestdlib/utils/prototestwarning)input_urischange, a fork with File and Dir overrides read each action's inputs from its own path and applied both.Companion SDK PR: flyteorg/flyte-sdk#1708 (the controller applies the overrides).
🤖 Generated with Claude Code