Skip to content

Regeneration onto a fixed release runs inside the agent session, so a boot-blocking defect leaves consumers with no self-heal path #1285

Description

@tend-agent

A defect that stops the agent booting takes out the mechanism consumers would use to pick up its fix, so every consumer stays broken until a human runs uvx tend@latest init by hand. Today's bubblewrap outage is the third instance of the precondition in nine days, and the first where a tend session in this repo could observe the whole thing.

The loop

macros.yaml.j2 states the design: the action ref pins to an immutable release tag, and "consumers move forward via the nightly uvx tend@latest init regen pulling a newer generator, which restamps a newer tag as a reviewable diff in their own repo." That regen runs inside the agent turn — nightly/SKILL.md has the nightly session call nightly_workflow_update.py prepare, edit the prepared worktree, run uv tool run tend@latest init in it, and ship — so it needs a booted agent.

When the defect is in the boot path itself, the agent never starts, the nightly never regenerates, and the pin never moves. There is no other path: tend-nightly is the only workflow that regenerates, and nothing in the generated workflows runs init outside the sandbox.

Why this isn't a one-off

Three fleet-wide boot failures since 2026-09-08, each killing sessions before the first turn:

When Cause Recovered by
2026-09-08 0.2.2 shipped shared/steps/install-uv.sh non-executable across the sandbox boundary 0.2.4 + hand regeneration
2026-09-08 0.2.3 published an action cache missing shared/steps/sandbox_runtime.mjs 0.2.4 + hand regeneration
2026-09-17 bubblewrap 0.9.0-1ubuntu0.2 (CVE-2026-87766) made a 0o711 bind-destination parent unopenable — #1277, #1279 0.2.10 + hand regeneration (#1283)

The first two are release defects, where "don't ship it" is at least a conceivable answer. The third isn't: nothing in tend changed. The Ubuntu archive moved under a running fleet, so no amount of pre-release care would have prevented it, and rolling the pin backwards wouldn't have helped either — Ubuntu keeps one version per pocket and the superseded package was already gone.

In each case this repo recovered in under an hour because a human was watching. A consumer whose maintainer is asleep recovers whenever they next look at their Actions tab.

What it costs

Little in tokens — these failures die before the first turn, so today's eleven cost $0 in model spend. The cost is the outage itself: from the first confirmed failure to the regeneration onto 0.2.10, every tend-review, tend-triage, and tend-notifications event was dropped rather than deferred. #1279 sat unanswered for 49 minutes; #1282's review died and was recovered only because this daily sweep looked for it; #1277, #1281, and #1283 — including the release that ended the outage — merged with no bot review at all. In a consumer repo the same window is however long it takes someone to notice, and the outage tracker filling with rows is the only signal, in a repo whose whole point is that nobody has to watch it.

#1282 closes the detection half of this — a scheduled ci run watching from outside the sandbox. The recovery half is still open, and detection doesn't imply it: a consumer can know their bot is down and still have no automated way to move the pin.

The decision

Three shapes, and which is right is a call about the security model rather than a bug fix, which is why this is an issue and not a PR:

  1. A runner-side regen step. tend-nightly runs uvx tend@latest init on the runner before the agent action, and opens the PR itself when the generated files change. Cheapest to build and it works with the agent completely broken — but it's a second regeneration path racing the agent's, and CLAUDE.md is explicit that deterministic YAML steps shouldn't reimplement work that happens inside an agent run. The counter-argument is that this is the one case where that rule's premise fails: the agent run is what's broken.
  2. A separate minimal self-update workflow, with no bot token beyond contents: write and no consumer code in scope. Keeps the nightly's regen where it is and adds a path that survives the sandbox. More surface than (1).
  3. Accept it. A fleet-wide boot failure is rare and loud, and arguably should have a human in the loop before every consumer's workflows are rewritten automatically. If this is the answer, the gap is documentation rather than mechanism — the outage tracker body could name uvx tend@latest init as the recovery when every row is a pre-agent failure, so the consumer maintainer who does look has the command without reading tend's source.

Option 3 is a legitimate answer and may be the right one; what's not defensible is the current state, where the loop exists and nothing says so.

How this was derived

From the tend-review-runs sweep over 2026-09-17T07:49:45Z → 19:41Z. This repository's run history brackets the break precisely: the last agent boot to succeed was tend-notifications 35246533937 at 16:25:52Z, and the first failure was tend-review 35261197467 at 18:49:47Z. Eleven runs failed at bwrap: Can't mkdir parents, all recorded in #1278. The nine tend-notifications ticks between 16:25 and 18:49 all report success because their pre-check early-exits before installing bubblewrap, which is why the Actions tab stayed green throughout.

The 0.2.2 and 0.2.3 rows are from the 2026-09-09 review-runs entry in #1125, which classified them Critical and structural and recorded 24 zero-turn failures across the two.

Note that #1277's "16:16" and "17:26" install timestamps are not from runs in this repository — no run started here between 16:03 and 16:25, or between 17:19 and 17:30 — so they are presumably from a consumer's history. The bracket above is this repo's own.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    review-runsFindings from review-runs skill

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions