Bug description
autoloop verify <project> <run-id> forgets run-scoped acceptance.verify_cmds supplied with autoloop run --set ... and can report an acceptance-verified run as unverifiable.
Steps to reproduce
- Launch a run with a run-scoped override:
autoloop run autocode "task" \
--set 'acceptance.verify_cmds=python3 -m compileall -q .,python3 -m pytest -q'
- Allow it to complete. The registry can show:
{"status":"completed","acceptance_verified":true,"outcome":"verified"}
- Run independent post-fire verification:
autoloop verify /path/to/project <run-id> --json
Actual:
{"reconcile":"unverifiable","ranChecks":0,"failures":[]}
Root cause
In autoloop-cli/dist/commands/verify.js, post-fire verify does:
const cfg = config.loadProject(record.project_dir || projectDir);
const cmds = postFireCheckCommands(config.getList(cfg, "acceptance.verify_cmds"), ...)
The effective run-scoped overrides are not persisted in the run record, so a later process reloads only project/preset config and loses the commands used by the run.
The compatibility path is confirmed: setting AUTOLOOP_CONFIG to a temporary TOML containing the same acceptance.verify_cmds makes post-fire verify return confirmed.
Expected behavior
A later autoloop verify should reuse the exact effective acceptance contract from that run without requiring callers to resupply it.
Suggested solution
Persist the effective post-fire acceptance commands and timeout in run metadata/journal at launch, then have verify prefer the recorded immutable contract. Avoid reinterpreting mutable project config for historical runs.
Also consider preserving commands as a true array: AutoLoop 0.10.1's string-list parser splits commas inside command strings, so CSV serialization is lossy.
Environment
- AutoLoop 0.10.1
- Node 22.23.1
- Linux
Bug description
autoloop verify <project> <run-id>forgets run-scopedacceptance.verify_cmdssupplied withautoloop run --set ...and can report an acceptance-verified run asunverifiable.Steps to reproduce
{"status":"completed","acceptance_verified":true,"outcome":"verified"}Actual:
{"reconcile":"unverifiable","ranChecks":0,"failures":[]}Root cause
In
autoloop-cli/dist/commands/verify.js, post-fire verify does:The effective run-scoped overrides are not persisted in the run record, so a later process reloads only project/preset config and loses the commands used by the run.
The compatibility path is confirmed: setting
AUTOLOOP_CONFIGto a temporary TOML containing the sameacceptance.verify_cmdsmakes post-fire verify returnconfirmed.Expected behavior
A later
autoloop verifyshould reuse the exact effective acceptance contract from that run without requiring callers to resupply it.Suggested solution
Persist the effective post-fire acceptance commands and timeout in run metadata/journal at launch, then have
verifyprefer the recorded immutable contract. Avoid reinterpreting mutable project config for historical runs.Also consider preserving commands as a true array: AutoLoop 0.10.1's string-list parser splits commas inside command strings, so CSV serialization is lossy.
Environment