Skip to content

Batch action application with per-action veto - #528

Open
ndisidore wants to merge 20 commits into
mainfrom
feat/action-apply-through-foundation
Open

ndisidore wants to merge 20 commits into
mainfrom
feat/action-apply-through-foundation

Conversation

@ndisidore

@ndisidore ndisidore commented Sep 18, 2026

Copy link
Copy Markdown
Member

This adds the shared applyActionsThrough contract and replaces the overseer’s auto-approval drainer with ActionSyncDriver. The driver resolves pending actions in ordered passes, stages vetoes, records failures, and falls back to the existing per-action calls for gatekeepers that have not migrated. The public approveAction and rejectAction RPC signatures remain unchanged but are now depreciated.

Frontend behavior is intentionally minimal and the primary difference is that action cards now render the gatekeeper's failure reason, and there is distinction between "Denied" and "Invalidated" (new label); the existing per-action controls stay in place, and batch UI application work will follow separately.

Design Decisions

  • One ordered prefix per call. Frontier and its vetoes in a single RPC. Leaves ordering and cascade reporting where the provider can answer them.
  • Checkpoint at the provider boundary. Vetoes durable as vetoPending before the call, cleared only on acknowledgement. Same discipline in the legacy path, one checkpoint per acknowledged call.
  • Ordering is the provider's obligation. Retained actions published in ascending order, so a frontier means what it says. Cheaper than a second reconciliation system in the kernel for late IDs.
  • A stop leaves the action pending. Bounded, display-safe reason, no resolver attribution. The gatekeeper said why it stopped, not whether the work landed. That reason then gates every unattended path: rules skip it, the card withdraws "Always approve", and a stop outranks a gate when both hold.
  • Pack authority expires with the invocation. Disposable builder over the authorized pushes only. Cache stays readable for simulation, can't mint packs.
  • Legacy fallback narrow and temporary. Probe only on the runtime's missing-method error; coded and ordinary failures never replay. Deletable once providers migrate.

Devin Review

@github-actions github-actions Bot added workshop/frontend Changes to the Workshop frontend kernel Changes to the Workshop kernel workshop/shared Changes to shared Workshop APIs labels Sep 18, 2026
@github-actions

Copy link
Copy Markdown

Preview: pr528-feat-action-a-7a61eb44

https://pr528-feat-action-a-7a61eb44-router.cloudflare-os-previews.workers.dev

Dashboard · deleted when this PR closes

devin-ai-integration[bot]

This comment was marked as resolved.

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Sep 18, 2026

Copy link
Copy Markdown

Submitted 3 actionable inline findings.

github run

Comment thread packages/workshop-shared/src/api.ts
@ndisidore
ndisidore force-pushed the feat/action-apply-through-foundation branch from 69657eb to ef484c8 Compare September 18, 2026 14:22
devin-ai-integration[bot]

This comment was marked as resolved.

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

This comment was marked as resolved.

@ask-bonk

This comment was marked as outdated.

devin-ai-integration[bot]

This comment was marked as resolved.

@ask-bonk

This comment was marked as outdated.

@ask-bonk

This comment was marked as outdated.

@ndisidore
ndisidore force-pushed the feat/action-apply-through-foundation branch from ee99ed3 to 384ff68 Compare September 18, 2026 16:11
@ask-bonk

This comment was marked as outdated.

@ask-bonk

This comment was marked as outdated.

@ndisidore
ndisidore force-pushed the feat/action-apply-through-foundation branch from 384ff68 to 81c9da4 Compare September 18, 2026 17:15
@ask-bonk

This comment was marked as outdated.

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Sep 18, 2026

Copy link
Copy Markdown

Submitted 4 actionable inline findings.

github run

@ndisidore
ndisidore force-pushed the feat/action-apply-through-foundation branch from 81c9da4 to 395cf4c Compare September 18, 2026 23:45
devin-ai-integration[bot]

This comment was marked as resolved.

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Sep 19, 2026

Copy link
Copy Markdown

Submitted 2 actionable inline findings.

github run

@ask-bonk

This comment was marked as outdated.

@ask-bonk

This comment was marked as outdated.

@ask-bonk

This comment was marked as outdated.

A cleanup pass over the branch, removing checks whose failing branch no code
path can enter, and docs that restated a rule already stated where it belongs.

`isMethodMissing` guarded against a coded Git pack error whose message
resembled workerd's method-missing prose. The four pack messages are fixed
strings that contain no such text, so the guard, its import, and the test
that had to hand-mutate an error's message to reach it are gone.

`GitPackBuilderImpl` rechecked `action` and `gatekeeperId` on a record it
looked up by a map keyed on exactly those values, populated from this
connection's own pending plan; both fields are immutable on a record. The
dispose-time map clear was dead behind the `#active` flag. The fourth code,
GIT_PACK_ACTION_DECLARES_NO_PUSH, is folded into "not authorized": the
constructor filters on `pushedCommits?.length`, so an empty declaration never
enters the map, and the contract already tells implementers to omit the field
rather than pass `[]`. The next commit closes the one producer that could
forward `[]`.

`applyThrough` re-read the boundary's connection inside the queue and compared
it to the pre-queue read; a record never changes connection. `reject()`
re-read the record after the gatekeeper call, but the decision queue holds
across that await, so nothing can decide it meanwhile. The legacy veto loop
logged and rethrew a failure the run loop already logs and batch callers
already receive.

Docs: the publish-order rule was stated three times; `applyActionsThrough` now
points at the normative text on `submitAction`. The hand-written function types
on the pack error exports now match their siblings in api.ts.

Tests: the two-scenario pack test is split so a failure names its scenario, the
frontend card renderer is shared between the pending and resolved cases, and
the RPC-safe `expectGitPackCode` helper gains a comment saying why it must not
become `expect().rejects` -- handing an RPC-stub promise to vitest leaves an
unhandled rejection behind in workerd.
The kit forwarded a provider's `pushedCommits` whenever it was truthy, and `[]`
is truthy. A `describe()` that computed "commits to push" and found none would
put `pushedCommits: []` on the wire, which the overseer reads as a push
declaration: it verifies no ancestry and marks no objects for it, and the pack
builder refuses to build for it, so the action could only stop at apply time
with a message that did not name the mistake.

Filter on length instead of presence. An empty list is "no git", the same as no
key, and now produces the same wire shape. The kit was the only producer that
could forward `[]`; the GitHub gatekeeper always declares one commit directly.
The existing no-key test is parameterized over the empty list so the truthy
hole stays pinned as observable wire behavior.
The offer's own comment says it appears only when enabling a rule would
actually apply this action, and it checked three of the conditions for that: a
tagged action, on a connection, that the gatekeeper marked auto-approvable. A
recorded failure is a fourth, added later -- it disqualifies the action from
the rule path, since nothing unattended may retry a side effect whose outcome
the gatekeeper never confirmed. So on a stopped card the button still appeared,
the confirm dialog still promised application, and enabling the rule left that
action pending, with any agent turn awaiting it still suspended.

Check the failure in both gates. The rule stays creatable from the
auto-approval panel and Approve still works on the card, so nothing is lost
except a promise that could not be kept. The backend is untouched: refusing to
replay a stopped action unattended is the behaviour the offer was misreporting,
not a bug in it.
@ndisidore
ndisidore force-pushed the feat/action-apply-through-foundation branch from 09b2925 to 2ff9a51 Compare September 21, 2026 17:19
@cloudflare cloudflare deleted a comment from ask-bonk Bot Sep 21, 2026
@cloudflare cloudflare deleted a comment from ask-bonk Bot Sep 21, 2026
@ndisidore
ndisidore added this pull request to stack #543 September 21, 2026 21:10
@ndisidore
ndisidore removed this pull request from stack #543 September 21, 2026 21:10
@ndisidore
ndisidore added this pull request to stack #544 September 21, 2026 21:10
@ask-bonk

ask-bonk Bot commented Sep 21, 2026

Copy link
Copy Markdown

@ndisidore Bonk workflow failed. Check the logs for details.

View workflow run · To retry, trigger Bonk again.

`ActionSyncStorage` restated the overseer's action schema by hand: a collection
plus the two indexes the driver reads. A hand-mirrored shape can drift from the
thing it mirrors, and the tests already build their storage from the production
`makeOverseerStorage`, so the interface bought nothing the real type does not
give. The `applyLegacyAction` hook existed for a narrower reason: the driver
had no way to build an action-scoped `GitCacheImpl`, so the overseer supplied
the whole call instead of just the cache.

Derive the storage type with `Pick`, and give `createGitCache` the optional
action id the legacy path needs so the driver can make its own call. Two hooks
become one and the legacy call site now reads like the native one above it.
Tests lose a seam they only had to route around: `makeDriver` no longer
reimplements the apply, `putAction` defers to the shared fixture, and
`rejectionOf`/`rejectBatchProbe` replace the try/catch and the hand-written
missing-method TypeError that several suites had each spelled out.
The chat card and the Activity row each decided independently whether to offer
"Always approve this type", with the same four-part condition written out
twice. Keeping two copies in step by hand is what let them drift: when a
recorded failure became a reason to withhold the offer, both copies needed the
new clause, and the type the confirmation dialog consumes was declared inline
in both files as well.

`autoApproveTargetOf` now owns that decision, beside `actionStatusLabel`, which
is the same move for the same reason. Each caller narrows to an action entry
and asks. The rationale for each clause lives with the code that applies it
rather than in a comment duplicated next to each copy.
devin-ai-integration[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Sep 22, 2026

Copy link
Copy Markdown

Performance is inconclusive: comparable passes rose from 17/20 to 19/20. Project-doc improved 7/10→9/10 and ran 13.3% faster, while appointment-desk stayed 10/10 but slowed 32.4%; expense-ledger was incomparable. Project-doc more often kept placeholder details inside list items, avoiding baseline failures with bare labels. Ten trials per cohort leave substantial stochastic uncertainty.

github run

devin-ai-integration[bot]

This comment was marked as resolved.

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Sep 22, 2026

Copy link
Copy Markdown

Performance regressed on latency while correctness stayed unchanged at 23/30 passes. Mean duration rose 57.8% for appointment-desk and 16.7% for expense-ledger, while project-doc was flat. Appointment-desk added a model turn and more tool calls on average, with one candidate trajectory querying blueprints before creating the gadget; expense-ledger conversely reduced tool errors from 1.5 to 0.1. With only 10 trials per cohort, stochastic latency variation limits confidence.

github run

A gatekeeper that has already applied an action cannot honour a veto of it, and
the contract told it to ignore the veto silently. The caller then acknowledges
a rejection it never got: the record is written `rejected` for work the
provider executed, and because `persistRejected` also runs `clearPushMarks`, a
Git push additionally loses the `onRemote` proof that makes its objects
re-pullable. The gatekeeper knows which it is; nothing asked it.

Report those ids in `alreadyApplied` and reconcile them to applied. Reported in
the result rather than thrown, matching `invalidatedByVeto`: a throw would
abort every other veto and apply in the batch, and since the caller replays its
staged vetoes it would throw again on each retry and strand them. The record
keeps no resolver, because the pass that applied it lost its response before
recording one and this pass only knows the vetoer, and it sheds any `failure`
left by an earlier stop, since `ActionLogEntry.failure` is cleared when an
action applies and both cards render the note whatever the state. Only ids this
call actually sent are honoured, since vetoes beyond the frontier stay staged
and undelivered by design.

Inert until a gatekeeper implements `applyActionsThrough`, so populating it is
an acceptance criterion of the first native port rather than something today's
legacy path can exercise.
@ndisidore
ndisidore force-pushed the feat/action-apply-through-foundation branch from c6fe678 to 725b82d Compare September 22, 2026 15:04
@cloudflare cloudflare deleted a comment from ask-bonk Bot Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026

Copy link
Copy Markdown

Eval runs comparison

Baseline 87e09feb682f53ed1cd9b0f89cde03b3ed55cda9 vs candidate 36b7438c851b975d08908040e2ed4e04d6ba7743.

Task Model Baseline Candidate Pass-rate delta Duration delta Tool-error delta Cost delta
appointment-desk gpt-5.6-luna 9/10 (90.0%) 10/10 (100.0%) eval definition changed
expense-ledger gpt-5.6-luna 5/10 (50.0%) 7/10 (70.0%) eval definition changed
project-doc gpt-5.6-luna 9/10 (90.0%) 9/10 (90.0%) eval definition changed

Run · trajectories and raw results

@ask-bonk

ask-bonk Bot commented Sep 22, 2026

Copy link
Copy Markdown

Eval runs review

Measured. No cohorts are comparable; all three are marked “eval definition changed.”

github run

A veto the gatekeeper refuses because it had already applied the action
reconciles the record from rejected to approved, and `vetoRefused` tells the
caller why. Only `applyActionsThrough` reads it. `approveAction` and the
background rule passes both ride staged vetoes too, and neither reports
anything, so the card flips from Denied to Approved with nothing on it to say
the user asked for the opposite. Throwing from those routes is not the answer:
`approveAction` returns early once the clicked action reads approved, so the
check would have to precede that return and fail a click that succeeded, over a
reversal on a different card.

Mark the record instead, and give the marked state its own label. The flag
rides the same path as `cascadedFrom`, the label comes out of the one helper
both surfaces already share, and no route needs to learn to report anything.
`actionStatusLabel` now covers both states that would otherwise read as a
verdict nobody gave: taken down by an earlier rejection, and applied before a
rejection could land.
ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

This comment was marked as outdated.

@ask-bonk

ask-bonk Bot commented Sep 22, 2026

Copy link
Copy Markdown

Eval runs review

Measured. No cohorts are comparable; all three are marked “eval definition changed.”

github run

Three gaps around rejecting an action, each reachable from the card.

A veto the gatekeeper refuses as already applied reconciles the record to
approved, so the resume gate read it as an approval and could resume the turn
once its siblings applied. The user asked to stop; a late refusal does not
change that. The gate now treats `vetoRefused` like a rejection.

Rejecting an action also unblocks any rule-approved actions queued behind it,
but nothing ran a pass until the next unrelated trigger, leaving their turns
suspended. `rejectAction` now runs the same background pass-and-resume that
enabling a rule does, shared as `#applyDecidedInBackground`.

A cold reconnect re-fetched only pending cards, so a Denied card whose veto was
refused while disconnected kept reading Denied. It now re-fetches every card
that is not approved.
// Deny leaves the turn ended, like denyConnectionRequest. The rejected record also prevents a
// sibling approval from resuming this turn.
// sibling approval from resuming this turn. Rule-approved actions it unblocked apply now.
this.#applyDecidedInBackground(action.gatekeeperId);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Resume with the suspended turn's original user context. This new pass runs through the rejecter's OverseerClientInterface: if Alice's await-decision action is auto-authorized behind a blocker and Bob rejects that blocker, the action applies under Alice's saved rule, but #resumeSuspendedAgent() resolves the original model ID through Bob's #clientUser and attributes the continuation to Bob. Bob may not have that model, leaving Alice's chat suspended, or a matching ID can run with Bob's model config and credentials. Persist/use the suspended turn's initiator user rather than the collaborator who happened to trigger reconciliation.

@ndisidore ndisidore Sep 22, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm so Bob can resume Alice's suspended turn without deciding anything about it. That turn then runs on Bob's model record and Bob's user id. It needs a workspace with several builders who share a connection, a rule, and an awaited action.
main does't really do any tracking here: I could add this with about 15-20 lines of kernel code but for now it may be pragmatic to defer

@ask-bonk

ask-bonk Bot commented Sep 22, 2026

Copy link
Copy Markdown

Submitted 1 actionable inline finding.

github run

@ask-bonk

ask-bonk Bot commented Sep 22, 2026

Copy link
Copy Markdown

Eval runs review

Measured. No cohorts are comparable; all three are marked “eval definition changed.”

github run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gatekeeper Changes to a gatekeeper integration kernel Changes to the Workshop kernel workshop/frontend Changes to the Workshop frontend workshop/shared Changes to shared Workshop APIs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant