Skip to content

Decide: does reset_sandbox(force: true) survive #1768's reset fence, or does reaping replace it? #1925

Description

@jhgaylor

Two designs now answer the same question — "a reset fence must not be a dead end" — and they collided when #1732's stack was rebased onto main. One has to win before #1744 can merge.

main @ a7bfb8ba, and origin/feat/durable-turn-deadlines @ c0216625, both changed Conversations.reset_sandbox/2. The rebase fails on commit 1 of 21, in that function, and every other PR in the stack chains off it.

The same problem, twice

#1744 (ADR 0046) found that an unresolved bounded execution made reset_sandbox/2 answer :sandbox_mid_turn forever: awaiting_identity and uncertain are absorbing states, nothing revisited them, and _unsafe_sandbox_open?/1 blocks reset while either is set. Reset is the owner's documented recovery (#1071), so a provider that never named its session or never acknowledged a stop cost them that recovery permanently. Two exits were added:

  • ExecutionGuard._unsafe_retire_unresolved/2 — ages an obligation out after an hour, keeping last_error.
  • reset_sandbox(force: true) — resets past an unresolved execution and retires the journals bound to that machine, via _unsafe_retire_sandbox/1 + audit_retired/1.

#1768 (a7bfb8ba) independently found that a reset whose provider delete was never confirmed stranded the row, and answered it differently:

  • the decision is made on the row re-read under FOR UPDATE, not on the caller's struct;
  • only ready/suspended reset — pending and starting are now refused too, as the provision watchdog's business;
  • reset_requested_at is a durable fence: repeat resets answer :sandbox_reset_pending with no second delete, and so does anything that would re-use the machine;
  • two audit rows, sandbox.reset_requested when the fence commits and sandbox.reset only when the provider confirms;
  • and the escape, quoting the docstring: "The fence is not a dead end." A write that retires the row still goes through, so "an operator reaps it from /admin/sandboxes … Reaping is the supported way out of an unconfirmed reset."

The question

Does reset_sandbox(force: true) survive, or does #1768's reaping replace it?

Reaping already gives an operator a way past a stuck reset, and update_sandbox/2 deliberately lets a terminal write through the fence so that agent deletion and account deletion still complete. If that also counts as the way out of a stuck execution fence, then force: is a second lever for one job.

The counter-argument: they are not the same fence. :sandbox_reset_pending means Fountain asked the provider to delete and has not heard back. An open journal means a bounded command may still be running and its termination is unconfirmed — a different fact, on a different row, with a different owner (the deadline coordinator, which ages it out). Reaping is also an operator action from /admin/sandboxes; force: is available to the tenant who owns the machine. If the answer is "reap it", a tenant whose own conversation is fenced has to ask an operator.

Option A — reaping replaces force:

  • Delete the force: option, its docstring paragraph, and its two regressions in sandbox_reset_test.exs.
  • Keep _unsafe_retire_unresolved/2 (the ageing exit) — it is the coordinator's, and unaffected.
  • _unsafe_retire_sandbox/1 still has to be called where a reset does succeed, inside fix: fence sandbox resets until deletion succeeds #1768's locked decision block, so a destroyed machine's journals do not outlive it.
  • ADR 0046 records that the tenant-facing exit is the age, and the operator-facing one is reaping.
  • Journal bounded turn deadlines and remote stop intent #1744 gets smaller. Fewer levers, one story.

Option B — force: stays

Either way, and independent of the answer:

State

All nine PRs (#1744#1752) are approved and green, and out of draft. #1744 is CONFLICTING; the other eight read MERGEABLE only because each is measured against its parent branch, so nothing lands until this is settled. #1754 stays a draft reference.

Blocks #1732.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:conversationsConversation lifecycle, turns, streaming, recovery and orchestration.area:sandboxSandbox providers, machine identity, capabilities and lifecycle.type:questionA question or investigation needs an answer.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions