Skip to content

fix(#4539): surface artifact contract in spawn prompt - #4575

Merged
chernistry merged 3 commits into
sipyourdrink-ltd:mainfrom
jm27:fix/4539-artifact-contract-prompt
Aug 25, 2026
Merged

fix(#4539): surface artifact contract in spawn prompt#4575
chernistry merged 3 commits into
sipyourdrink-ltd:mainfrom
jm27:fix/4539-artifact-contract-prompt

Conversation

@jm27

@jm27 jm27 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Problem

Issue #4539: an artifact-mode task declares a full contract (kind, output_path, acceptance criteria) but the spawn prompt never surfaces it. Agents only learn the contract from operator free-text; when that's skipped, completion fails on criteria the agent never saw.

Fix

Adds a shared render_artifact_contract(tasks) helper that renders the kind, the exact output path the verifier reads, and every declared criterion. It reads the same task.artifact_spec object and the same artifact_output_path / is_artifact_mode resolvers as the completion path, so prompt and verifier cannot drift (no second parse).

Wired into both render paths:

  • spawner_core._render_prompt_with_receipt (the live production path)
  • spawn_prompt._render_prompt (kept in sync per that file's convention)

Gated so a code_diff batch's prompt is byte-unchanged.

Closes #4539.

Tests

Three new tests in tests/unit/agents/test_artifact_contract_prompt.py:

  1. test_artifact_task_prompt_names_kind_path_and_criteria
  2. test_code_diff_task_prompt_is_unchanged
  3. test_prompt_and_verifier_consume_the_same_spec_object

All pass (pytest tests/unit/agents/test_artifact_contract_prompt.py → 3 passed). Existing prompt-coherence tests unchanged (29 passed). Ruff clean.

Artifact-mode tasks complete on a signed lineage receipt over their
produced artifact, but the spawn prompt never surfaced the contract the
completion side enforces: kind, output path, and acceptance criteria.
Agents could only learn the contract from operator free-text, and when
that was skipped, completion failed on criteria the agent never saw.

Add a shared render_artifact_contract() helper that reads the same
task.artifact_spec object and the same artifact_output_path /
is_artifact_mode resolvers as the completion path, so prompt and
verifier cannot drift. Wire it into both render paths (spawn_prompt and
the live spawner_core), gated so a code_diff batch's prompt is
byte-unchanged.

Closes sipyourdrink-ltd#4539

@chernistry chernistry left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against the code — this is right, and right for the reason that matters: the prompt and the verifier read the same artifact_spec through the same artifact_output_path / is_artifact_mode resolvers, so there is no second parse to drift. That is the part that would have rotted if the contract had been re-rendered from its own copy of the spec.

Checked the two things worth checking:

  • spawner_core._render_prompt delegates to _render_prompt_with_receipt (spawner_core.py:1430–1439), so the test assembles the live production path rather than a parallel one. The assertion artifact_output_path(task) in prompt is what makes it a real test and not a string check.
  • The gate is on is_artifact_mode, which resolves through artifact_spec.kind and defaults to code_diff, so an ordinary coding task's prompt is byte-unchanged — covered by test_code_diff_task_prompt_is_unchanged.

One thing you may be wondering, since the PR touches both render paths: only the spawner_core path is exercised by the tests. That is fine as it stands — spawn_prompt._render_prompt is kept in sync by that file's own convention and nothing spawns through it in production. No change requested.

Approved, auto-merge armed. It will enter the merge queue once the 3.13 shards finish.

If you want another one in the same area: #4571 — the agent timeout extension never reaches the spawned process. It is fully diagnosed and small:

  • tuning.orchestrator.max_agent_runtime_s does reach OrchestratorConfig.max_agent_runtime_s (core/models.py:1603, default from _default_max_agent_runtime_s at models.py:44).
  • Its only consumer is the extension logic at core/agents/agent_lifecycle.py:2834.
  • Every real adapter.spawn(...) call site omits timeout_seconds, so adapters/base.py:42 DEFAULT_TIMEOUT_SECONDS = 1800 arms the threading.Timer instead. The configured value cannot take effect no matter what an operator sets.

The slice: thread the resolved runtime into the spawn calls so the Timer is armed from config. Proven by a test that spawns with a non-default max_agent_runtime_s and asserts the adapter received that timeout — not by reading the config back. Out of scope: the extension logic itself, and any change to the 1800 s default for callers that pass nothing.

I have assigned it to you. If it turns out bigger than it looks, say so in the thread and I will slice it further.

@chernistry
chernistry enabled auto-merge August 25, 2026 21:09
The prompt renderer now reads the task's artifact contract to decide whether
the task completes on a signed receipt or on a commit. These two tests build
their task with a bare MagicMock, which answers that question with a Mock -
not `code_diff` - so the renderer took them for artifact-mode tasks and
raised while resolving an output path that does not exist.

The mock already enumerates every field the renderer reads; this adds the one
the renderer started reading.
@chernistry

Copy link
Copy Markdown
Collaborator

Shard 4 was red on test_lessons.py, not on your change's logic: those two tests build their task with a bare MagicMock, so is_artifact_mode saw a Mock instead of code_diff and the renderer went looking for an output path. Pushed a commit to your branch giving the mock an ArtifactSpec() — the mock already enumerates every field the renderer reads, this adds the one it started reading. CI is re-running.

Worth knowing for next time: your first push sat for a few hours with no checks at all. That is the first-time-contributor workflow gate, not your PR — a maintainer has to release each run. Ping me here if it happens again and I will clear it.

@chernistry
chernistry added this pull request to the merge queue Aug 25, 2026
Merged via the queue into sipyourdrink-ltd:main with commit a35044a Aug 25, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agents are never shown the artifact contract their task declares

2 participants