Skip to content

Latest commit

 

History

History
337 lines (272 loc) · 24.5 KB

compute.md

File metadata and controls

337 lines (272 loc) · 24.5 KB

ComputeDriver contract

Overview

ComputeDriver prepares and removes Namespace infrastructure and runs Agent revisions. OCC selects one Driver per Installation, authorizes operations, and stores immutable revision configurations. Compute owns gateway, workload identity, routing, activation, and readiness; its backend owns underlying resources. A selected SandboxDriver can create a dedicated Harness workload.

See Driver selection for supported combinations and package trust, the feature matrix, and current versus planned placement.

Interface

The shared contracts define the types. Every ComputeDriver has an id, implementation, and capability: "compute".

Optional getRuntimeImages(revision) returns {workload, container, image, imageId, commit, openclawCommit} entries for containers owned by that Agent revision. OCC requires exact Agent read authority and uses the Driver recorded in the selected Agent revision. runtime-images reports undeployed without a selected Agent revision and unsupported when the Driver omits this method.

Commits must be full lowercase Git SHAs; missing IDs or provenance remain null. Results exclude separate Sandbox Driver workloads. Neither Docker nor Kubernetes derives commits from a moved tag. Kubernetes inspects revision-owned Pods and binds its private metadata read to the Pod UID and running container ID; both commits apply only to containers with that image ID.

Optional discoverHarnessModels({provider, apiKey}) returns native model IDs and names without persisting credentials. OCC checks Agent creation authority first. Bundled Kubernetes and Docker call the official OpenAI and Anthropic model-list APIs with a 10-second timeout, a 4 MiB limit per response, at most 10 pages and 10,000 models, and no redirects. Discovery requires OCC API egress to each provider; it neither provisions runtime credentials nor proves model compatibility. Unsupported or unavailable discovery permits manual model entry.

The bundled Codex OAuth device-login implementation is Experimental. Optional startHarnessDeviceAuthorization(harnessId) returns a public challenge and opaque private state; pollHarnessDeviceAuthorization(privateState) returns pending or a native credential bundle. OCC owns authorization, scope, and Secret custody. See the device login flow.

Core lifecycle operations

Required method What it does
ensureNamespace(namespace) Prepares or checks Namespace infrastructure; returns namespaceReady. Runs before an Agent exists; do not require or guess its ID. A failure may add a reason of at most 256 characters for the worker log, using only letters, digits, spaces and . _ : / @ -; it must not carry provider output or another tenant's values.
deleteNamespace(namespace) Returns namespaceDeleted after supported teardown. OCC deletes only empty Namespaces. Without an approved backend deletion path, fail without deleting the physical namespace or Agent resources.
prepareRevision(revision, context?) Creates or reuses the Agent gateway and prepares the configured Harness workload. Returns ready for that Namespace, Agent, and revision, plus optional plugin warnings. ready: false stays pending, with an optional pendingReason; OCC rejects an invalid result.
stopRevision(revision) Removes inbound routing and stops this revision, including applicable hooks and Sandbox cleanup. Safe to repeat; retains snapshots, runtime credentials, workspace data, and other persistent Agent state.
retireRevision(revision) Revokes workload access, then stops the workload and requests applicable Sandbox cleanup. Preserves a replacement's Agent gateway.

Namespace results can mark a failure retryable or permanent; success requires a true flag and no failure. Methods without return values must reject incomplete work. Revision context is optional in TypeScript; the worker supplies it after authorization.

prepareRevision throws ComputePreparationFailedError for terminal initialization. OCC atomically fails deployment with durable stopRevision cleanup. Restart resumes cleanup, not preparation; codes exclude credentials.

Optional additions

Method or declaration When it is needed
bindAgent({ namespace, agent }) Receives the approved Namespace, Agent, and ServicePrincipal before revision work; may be asynchronous. Failure prevents further runtime work in that attempt.
validateHarnessAuth, validateConfiguration, validateGatewaySettings, validateWorkspaceSetup Side-effect-free admission checks at save, provisioning and deployment. Deployment and provisioning require validateHarnessAuth.
activateRevision(revision, context?), deactivateRevision(revision) Production startup requires both. The worker also calls activation if a development Driver provides it. See revision stages.
setLifecycleDrivers(drivers) Startup requires it when another selected Driver provides Compute hooks.
describePrepareRevisionFailure(error) Returns validated safe fields for rejected preparation. OCC attributes them to the Agent revision without serializing the raw error. Omission preserves generic retries.
resolveSandboxNamespace, withdrawCredentialSource Credential Gateway hooks for registration and withdrawal.
withdrawHarnessExposure(revision, signal) Withdraws a native worker's provider exposure outside a pass. The worker calls it at start for selected Agent revisions and failed candidates, with at most five logged attempts per Agent revision and worker start.
activationOrder, maintenanceIntervalMs Control activation timing and optional maintenance.
requiresStoppedPredecessors(revision) A side-effect-free declaration, derived from the saved Agent revision, that opts into exclusive replacement.

Optional startup preflight

preflight() checks dependencies before production startup completes; a thrown error blocks startup. The API and worker log warnings as compute.preflight-warning and continue; each has a stable code and a log-safe message. Bundled Kubernetes Compute requires it in production.

Optional gateway endpoint resolution

getGatewayEndpoint(revision) returns a private WSS address from trusted Driver settings and approved resource IDs, or undefined if gateway access is unsupported. OCC authorizes Agent access and reads its selected revision first. This method neither checks readiness, authorizes callers, grants backend route permissions, nor saves URLs in Agent Configuration. Connection errors are dependency failures.

Workspace-file access and the opt-in Agent native admin UI require this endpoint. Kubernetes implements private routes.

Optional initial runtime credential provisioning

getAgentRuntimeCredentialStatus(binding) returns transportConfigured when complete generated transport credentials are stored. provisionAgentRuntimeCredentials(binding, input) accepts an empty object and creates those credentials. The caller holds Namespace and Agent locks, requires a ready Namespace with no earlier revision, and passes approved identities, never storage names. Missing methods fail. External writes can survive database or audit failure.

Agent provisioning also requires validateAgentProvisioning({ executionMode, configuration }). A thrown ComputeGatewaySettingError reaches the caller as a 409 naming the setting, and the worker stores it as a permanent PROVISIONING_REJECTED. DependencyUnavailableError stays a retryable dependency failure. Any other error becomes a fixed 409, with the first 512 characters of its message logged as reason.

With requiresAgentRuntimeCredentials: true, OCC checks stored status and creates missing transport credentials before the first revision; later revisions cannot regenerate them.

deleteAgentRuntimeCredentials(binding) is the idempotent teardown counterpart. During Agent deletion, the worker calls it after retiring every revision and before removing the Agent's database identity. Kubernetes Compute deletes the Agent's workspace setup Secret, transport Secret, and owned claims, which revision retirement retains; absence is success. Namespace-owned Harness model authentication survives Agent deletion. A Driver that supports provisioning but not deletion fails Agent deletion permanently on its first worker attempt; one with neither method is unaffected.

Startup failure evidence

ComputeReadiness.runtimeFailure optionally reports a startup failure with validated fields for the observed Agent revision ID: safe component, check, checkedAt, code, and optional cause fields, never credentials or provider errors. An unavailable or untrusted observation omits it.

Runtimes hold published evidence until restart, so the worker fails at once with RUNTIME_ plus the code (RUNTIME_CPU_STARVED for MODEL_PROBE_CPU_STARVED; RUNTIME_STARTUP_FAILED for UNAVAILABLE and INCOMPATIBLE_RESPONSE), or WORKSPACE_SETUP_REFUSED for a refused setup; other codes persist at the convergence deadline. The deployment status API returns saved evidence without invoking Compute.

Optional runtime diagnostics

After OCC authorization, diagnoseAgentDeployment(binding) returns current checks for the requested Agent revision ID using its approved Namespace and Agent. The Driver maps native evidence to component, check, state, nullable checkedAt and optional code fields. It verifies runtime identity, bounds size and time, and omits credentials, provider output, logs. OCC rejects mismatched revisions, invalid timestamps, over 32 checks and unsupported states; values outside packages/contracts/src/runtime-failure.mjs read UNRECOGNIZED.

The call does not update deployment work, rerun the startup probe, send messages, or prove a model response. Missing support returns dependency unavailable; Drivers unable to collect safe evidence omit the method.

Optional runtime status and logs

describeAgentRuntime(binding, signal, options) returns Pod status, restarts, the newest 100 Events per Pod and log sources of a revision, or one source without Events. A terminated container reports its current exit as lastTermination; otherwise it reports the prior exit. readAgentRuntimeLogs(binding, request) returns raw lines from a listed Pod within the request's line and byte limits. Drivers re-check ownership and raise RuntimeLogsForbiddenByClusterError for a cluster 403; OCC redacts and bounds output. Without them, or with runtimeLogging: "driver", both routes answer 501. Kubernetes log reads can return no lines during initial Pod/container preparation; see log availability. Kubernetes normalizes RFC3339 timestamp offsets to UTC without reducing fractional precision.

Runtime logging ownership

Omitting runtimeLogging or setting it to "platform" uses the bundled Harness logging policy. With "driver", OCC preserves native logging after any Sandbox changes, validates it through ConfigurationDriver, and records it in the immutable revision. Compute must apply that configuration and reject unsupported changes. The Driver operator owns log destinations, credentials, redaction, access, and delivery; the bundled Collector's privacy and export guarantees do not apply.

The gateway and Harness emit their own logs in either mode. OCC does not transport them, and neither OCC nor Compute streams, stores or exports them; authorized readers fetch redacted pages on demand through the Driver, with a 512 KiB response limit. OCC process logs, lifecycle results and audit records use their own paths.

IAM

OCC authenticates callers and asks the selected IAMDriver to authorize each resource operation. The worker rechecks the original caller's authority before work and stops if access is denied or revoked, or IAM is unavailable. Backend credentials cannot replace platform authorization. Compute cannot choose a different Principal, Namespace, Agent, or revision.

Reading initial credential status requires Agent read; provisioning, including automatic creation before the first revision, requires Agent read and operate. See authorization.

ComputeRevisionContext.harnessAuth holds, without credential values, the approved API-key source and its current backend reference, the managed-account credential reference and private Backend binding, the current credential source record, or { method: "runtime" } for operator-managed authentication. The separate secretEnvironment holds Configuration bindings for gateway credentials. Deliver model credentials only to the selected Harness workload. Channel tokens are ordinary Namespace Secrets referenced by Configuration secretBindings; never expose them in responses, Configuration, audit, logs, or errors. See the credential delivery flow. Installed Drivers run with control-plane privileges. Validating a package does not isolate untrusted code.

Lifecycle

Driver initialization and shutdown

At startup, each API or worker constructs its Driver from trusted Installation configuration, validates its identity and methods, attaches required hooks, and runs production preflight when required or provided. The interface has no initialize, dispose, or destroy method; process shutdown does not delete managed resources. See the Driver loading flow.

Production revision stages

OCC records the Compute identity, Harness placement, and Configuration in the immutable revision. Before dispatch, the worker checks that the selected Compute still matches, rechecks authorization, and resolves current credential references. By default, the previous route serves while the replacement prepares; activation checks that the selected Agent revision ID matches the expected ID, then switches the route so the candidate serves. Activation must be safe to repeat and requires the configured runtime to be ready and authenticated. Deactivation is a separate stage for an unpublished candidate; stopRevision stops execution.

With the default activationOrder: "afterCommit", the worker records the selected Agent revision before activating it, retries finalization when needed, and on the first production deployment deactivates an unpublished dedicated candidate. A Driver that keeps one stable Agent runtime can select "beforeCommit" to activate the candidate before publishing it and skip that initial deactivation. If a required stage becomes unavailable, the worker cannot proceed.

When requiresStoppedPredecessors(revision) returns true, the worker closes earlier credential sessions and calls stopRevision for every earlier snapshot, including failed candidates, before preparing that revision. Later passes re-stop after failures and doubling lease intervals. Stop must wait for resource release, preserve durable data, and be safe to repeat; a stop failure prevents preparation. The Driver owns backend-specific termination and Sandbox cleanup.

A newer saved Agent revision requiring exclusive replacement supersedes older reconciliation and maintenance, even while the saved selection points to its predecessor. Old passes cannot recreate competing runtimes. This mode accepts downtime without automatic rollback: deploy a higher Agent revision number.

SandboxDriver coordination

A selected Sandbox works only with bundled Kubernetes Compute. Compute isolates the Namespace, calls the Sandbox's optional Namespace setup, and either passes prepared workload inputs to a Sandbox that creates the dedicated Harness or creates it itself. It waits for the workload, activates its route and calls Sandbox cleanup before removing provider resources. See the Sandbox lifecycle.

Optional selected-driver hooks

Other selected Drivers may provide afterNamespacePrepared, beforeWorkloadStart, beforeWorkloadStop, and beforeNamespaceDelete. Hooks run in selection order; cleanup runs in reverse. Startup hooks affect the combined embedded workload or the dedicated Harness, never a dedicated gateway. Their opaque- environment accepts at most 64 values and cannot contain plaintext credentials or reserved names. Hooks cannot change images, commands, placement, networking, authorization, or revision data.

Hooks must be safe to repeat. Failed preparation undoes completed hooks; revocation failures wait for retry. After cancellation, rollback uses a separate cleanup signal that expires after five seconds. Hooks must observe it. See the hook execution flow.

Optional active-runtime maintenance

maintenanceIntervalMs must be a positive safe integer. After activation or maintenance, OCC atomically queues another pass for the same Agent, revision and original deployment Principal, which prepares and activates the revision again. Failed observations schedule another authorized pass without changing the selected Agent revision. Maintenance survives worker restarts and ends when a newer revision replaces or supersedes it. Without an interval, lifecycle work responds to events. New deployments have limited retries.

Plugin startup warnings

Readiness warnings contain an approved pluginId and either PLUGIN_INSTALL_FAILED or PLUGIN_AUTH_REQUIRED. The Driver returns ready: true only after safely disabling failed plugins with the rest of the runtime ready. OCC records warnings from a successful deployment while the worker still owns the job; the record is neither a live health check nor an acknowledgment, and restarting recalculates it. Missing or untrusted startup status cannot prove readiness. See Kubernetes startup status.

Limits

  • Implementations differ in topology, credentials, Namespace deletion, and private gateway access; see the feature matrix. The gateway and Harness need not share a cluster or a component that writes their resources.
  • Initial credential helpers cannot rotate credentials, manage model authentication, or prove that credentials work or workloads are ready.
  • Selecting a different Driver does not migrate revisions that recorded the previous Driver's identity.

Troubleshooting

Symptom What to check
Startup fails The selected Driver, required production stages, hooks, and preflight errors; preflight warnings alone do not block startup. Fix it and confirm startup.
Deployment fails before work is queued The Harness and authentication settings, and whether validateHarnessAuth exists. Fix the combination or Driver and confirm OCC creates a revision.
Revision stays unready The reported Namespace, Agent and revision, workload readiness and authentication, and plugin startup-status trust. Fix it and confirm readiness.
Cleanup or replacement stalls Revocation, hooks, and Sandbox cleanup; a missing workload alone does not prove cleanup succeeded. Fix it and retry until cleanup or activation completes.
Maintenance stops after a policy change The original Principal's authority and IAM availability. Restore the permission or start a newly authorized operation; confirm reconciliation resumes.
Credential setup partially fails Refresh stored status before retrying; confirm the required groups report configured. That does not prove the provider accepts them.

Implementations

Related