Skip to content

feat(dashboard): continue a finished run as a conversation - #56

Merged
lerogo merged 1 commit into
mainfrom
dev/0.1.7
Aug 20, 2026
Merged

feat(dashboard): continue a finished run as a conversation#56
lerogo merged 1 commit into
mainfrom
dev/0.1.7

Conversation

@lerogo

@lerogo lerogo commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

feat(dashboard): continue a finished run as a conversation

  • Resume a terminal run in place, reusing its round ledger
  • Claim operator messages at round start so a stop never delays them
  • Order the transcript by time, not by round
  • Drop the round copy of a discarded reply
  • Add --reasoning-effort per run and per role
  • Offer a force stop only after SIGTERM is ignored

feat(dashboard): continue a finished run as a conversation

- Resume a terminal run in place, reusing its round ledger
- Claim operator messages at round start so a stop never delays them
- Order the transcript by time, not by round
- Drop the round copy of a discarded reply
- Add `--reasoning-effort` per run and per role
- Offer a force stop only after SIGTERM is ignored
Copilot AI lite review requested due to automatic review settings August 20, 2026 12:35
@lerogo

lerogo commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator Author

@Upper9527

@lerogo
lerogo merged commit a1dd930 into main Aug 20, 2026
2 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the harness + dashboard lifecycle so a terminal run can be resumed “in place” (continuing on its existing round ledger), improves lifecycle idempotency across resume generations, and adds a unified --reasoning-effort surface (global + per-role) that is forwarded to supporting backends and reflected in Web metadata.

Changes:

  • Add in-place resume (“continue”) with a resume generation counter (resume_epoch) and generation-scoped lifecycle command IDs.
  • Claim queued operator instructions at round start and order the conversation transcript strictly by time.
  • Add reasoning_effort across CLI/config/supervisor/adapters and expose richer agent availability + reasoning metadata via the model catalog.

Reviewed changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/webapi/test_resume_routes.py Adds HTTP tests for /resume, idempotency behavior, snapshot fields, and extra-round validation.
tests/test_resume.py Adds unit tests for ledger restore, resume epoch merging, supervisor resume modes, and CLI --resume guard.
tests/test_resume_manager_loop.py Adds async tests validating manager-loop resume semantics (round numbering, prompt history, events, budget).
tests/test_reasoning_effort_chain.py Verifies reasoning-effort propagation/validation across adapters, CLI resolution, config, and supervisor role configs.
tests/test_opencode_adapter.py Updates OpenCode availability probing expectations and asserts tri-state availability.
tests/test_model_catalog.py Updates CLI stubs to return --version output consistent with availability probing.
tests/test_manager_hardening.py Ensures reopening a run withdraws published final responses the dashboard reads.
tests/test_guard_exclude_paths.py Adds tests ensuring repeatable CLI flags override (not extend) project-config lists.
tests/test_deepseek_harness_adapter.py Updates DeepSeek CLI probing expectations and asserts version/availability fields.
tests/test_codex_adapter.py Updates Codex CLI stub to emit a version string so probing marks it usable.
tests/test_agent_registry.py Adds tests for the new agent registry, probing tri-state availability, and effort discovery/validation.
src/lh_harness/webapi/snapshot.py Sanitizes reasoning_effort in role configs surfaced to the browser.
src/lh_harness/webapi/server.py Adds optional extra_rounds parsing, resume mode selection, and projects resume/lifecycle fields into snapshots.
src/lh_harness/utils/agent_cli.py Allows probing a caller-provided binary path to keep Web responses self-consistent.
src/lh_harness/supervisor/service.py Implements resume modes, generation-scoped lifecycle command IDs, unified worker launch transaction, and reasoning-effort forwarding.
src/lh_harness/supervisor/lifecycle.py Introduces resume_epoch helpers/constants and treats creating as an active lifecycle state.
src/lh_harness/model_catalog.py Refactors agent metadata to use agent_registry probes and exposes richer reasoning/availability payloads.
src/lh_harness/manager.py Implements ledger replay on resume, claims operator messages at round start, updates budget semantics, and withdraws discarded replies.
src/lh_harness/dashboard/state.py Adds approval support for optional extra_rounds with safe normalization and persistence.
src/lh_harness/dashboard/gate.py Enables budget-related approvals to collect and forward an operator extra-round grant.
src/lh_harness/config.py Adds project-config support for global + per-role reasoning_effort.
src/lh_harness/cli.py Adds CLI flags for reasoning effort, adds supervised-only --resume, and fixes repeatable defaults semantics.
src/lh_harness/agent_registry.py Adds a single declarative agent backend registry with probing, tri-state availability, and effort validation/discovery.
src/lh_harness/adapters/opencode.py Normalizes/validates reasoning-effort input via shared validator and supports new reasoning_effort param.
src/lh_harness/adapters/deepseek_harness.py Rejects reasoning effort explicitly for backends without an effort switch.
src/lh_harness/adapters/codex.py Forwards reasoning effort as a Codex config override (no CLI flag available).
src/lh_harness/adapters/claude_code.py Forwards reasoning effort via --effort and records it in episode metadata.
README.zh-CN.md Documents v0.1.7 resume-as-conversation + reasoning-effort and transcript/stop behavior changes.
README.md Documents v0.1.7 resume-as-conversation + reasoning-effort and transcript/stop behavior changes.
pyproject.toml Bumps version to 0.1.7.
frontend/web/src/style.css Adds styling for approval extra-round input and transcript expansion/clamping helpers.
frontend/web/src/api.ts Extends Web API types and adds payload support for approvals extra rounds and resume mode/options.
frontend/core/test/runFeed.test.ts Adds core projection tests for resume generation behavior, transcript ordering, approvals extra rounds, and abort command availability.
frontend/core/src/types.ts Extends snapshot/approval shared types with resume epoch + stop fields + extra rounds.
frontend/core/src/statusView.ts Adds “awaiting handoff” projection to avoid UI dead-air after approving while worker still reports waiting_approval.
frontend/core/src/runView.ts Adds sortTranscript to order the conversation strictly by time while keeping untimed entries anchored.
frontend/core/src/runFeed.ts Adds resume-epoch-aware snapshot preference/merging rules and preserves approval extra-round grants under stale frames.
frontend/core/src/commands.ts Updates command gating to keep /abort available while stopping and adds /new --effort parsing/validation.
Files not reviewed (1)
  • frontend/web/src/style.css: Generated file
Suppressed comments (1)

src/lh_harness/manager.py:1182

  • The budget extension can exceed MAX_ROUNDS: when round_index == MAX_ROUNDS, max(round_index + 1, min(round_index + extra, MAX_ROUNDS)) evaluates to MAX_ROUNDS + 1. That breaks the global round ceiling and can also discard the just-written terminal reply even though the run cannot be extended safely. Consider ensuring ctx.round_budget never exceeds MAX_ROUNDS and handling the edge case (already at the ceiling) by rejecting/overriding a continue decision instead of synthesizing an out-of-range budget.
        extra = _extra_rounds(decision.get("extra_rounds")) or max(1, ctx.config.max_total_episodes or 1)
        # Always grant at least one more round: clamping to MAX_ROUNDS must not
        # produce a budget below the current round, which would end the run
        # immediately after the operator asked to continue.
        ctx.round_budget = max(round_index + 1, min(round_index + extra, MAX_ROUNDS))

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/lh_harness/manager.py
Comment on lines +281 to +283
# After a resume ``max_total_episodes`` is the *additional* budget, so the
# effective ceiling continues from the restored rounds.
round_budget = round_index + max(1, config.max_total_episodes)
@TON14

TON14 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

@lerogo Following your note in #54 — here is a test report for the v0.1.7 reasoning-effort feature (a1dd930), run end-to-end against real providers on the three backends I have installed. Every run below is a full Manager→Executor→Auditor loop to completion, with the run artifacts inspected afterwards.

What works

  • claude_code (--reasoning-effort low): task completed and audited; all 10 episodes carry --effort low in their commands and "claude_reasoning_effort": "low" in metadata.
  • opencode (1.18.19, real DeepSeek API, --reasoning-effort deep where deep is a custom variant defined in the workspace opencode.jsonc): task completed and audited; all 10 episodes carry --variant deep and "opencode_variant": "deep". I also pointed OpenCode at a request-capturing endpoint to confirm the variant's options genuinely reach the provider: the request body carries reasoning_effort with the variant's value. Free-form variant names passing the charset check is exactly right for OpenCode, since variants can be user-defined.
  • deepseek_harness (@deepseek-ai/dsh 0.1.0-rc.7, real DeepSeek API, no effort set): base adapter works — task completed, audit passed.
  • Unit suite on a clean checkout: 404 passed, 2 skipped.

Two observations

1. An explicit role effort on a backend without a reasoning switch disappears silently. lh-harness run --agent deepseek_harness --executor-reasoning-effort high ... completes normally, but the value leaves no trace at all — not in the spawned commands, not in episode metadata, and no warning on the console. _resolve_role_reasoning_effort returns None for an unsupported backend before the adapter can object, so the adapter's own "Rejected rather than ignored" guard never fires. Dropping an inherited global value across a backend switch makes sense; dropping an explicitly typed --executor-reasoning-effort with no message might surprise the operator — a console warning (or an error for the explicit flag) would make the behaviour visible.

2. A claude_code typo runs the whole task at default depth. claude -p ... --effort ultrathink prints Warning: Unknown --effort value 'ultrathink' — ignoring it and using the default effort to stderr and exits 0, so in a harness run the warning is buried in episode stderr and every round quietly runs at the default. The ReasoningSpec already documents this as validation="silently_ignored" — surfacing that warning once on the run console at startup would close the gap cheaply.

Offer: dsh reasoning depth

DeepSeek Harness does document a reasoning dial: the llm-deepseek plugin takes reasoningEffort (low/high/max). I have a practically verified mechanism from my earlier branch: append an - id: llm-deepseek layer to the --patch overlay the runner already writes. Verified against the real API on dsh 0.1.0-rc.7 — dsh --dump-config shows the value merged into the plugin config, and a full MEA run carried it in every episode. If you'd like dsh covered, I'm happy to send a small PR shaped around the ReasoningSpec registry (transport/source fields and all); just say the word.

PS: Neither English nor Chinese is my first language, so I've been posting in both to make reading easier for the team — both halves are translations anyway. Tell me which you prefer for future messages, English-only or bilingual, and I'll stick to that.


@lerogo 根据你在 #54 的留言,这是 v0.1.7 思考强度功能(a1dd930)的测试报告:在我本地安装的三个后端上,对接真实服务端到端测试。以下每次运行都是完整的 Manager→Executor→Auditor 循环跑到结束,并在事后检查了运行工件。

正常工作的部分

  • claude_code--reasoning-effort low):任务完成并通过审计;全部 10 个 episode 的命令带 --effort low,元数据为 "claude_reasoning_effort": "low"
  • opencode(1.18.19,真实 DeepSeek API,--reasoning-effort deep,其中 deep 是工作区 opencode.jsonc 中自定义的 variant):任务完成并通过审计;全部 10 个 episode 带 --variant deep"opencode_variant": "deep"。我还让 OpenCode 指向一个记录请求的端点,确认 variant 的选项确实到达服务方:请求体中带有对应值的 reasoning_effort。字符校验放行自由命名的 variant 对 OpenCode 来说完全正确,因为 variant 可以由用户自定义。
  • deepseek_harness@deepseek-ai/dsh 0.1.0-rc.7,真实 DeepSeek API,未设置强度):基础适配器正常——任务完成,审计通过。
  • 干净检出上的单元测试:404 通过,2 跳过。

两个观察

1. 在没有思考强度开关的后端上,显式指定的角色强度会无声消失。 lh-harness run --agent deepseek_harness --executor-reasoning-effort high ... 正常跑完,但该值没有留下任何痕迹——生成的命令里没有,episode 元数据里没有,控制台也没有警告。_resolve_role_reasoning_effort 对不支持的后端直接返回 None,适配器里"Rejected rather than ignored"的防线根本没有机会触发。跨后端切换时丢弃继承来的全局值是合理的;但对操作者显式敲入的 --executor-reasoning-effort 无提示丢弃可能让人意外——加一条控制台警告(或对显式 flag 直接报错)就能让这个行为可见。

2. claude_code 的强度拼写错误会让整个任务以默认深度运行。 claude -p ... --effort ultrathink 只向 stderr 打印 Warning: Unknown --effort value ... 然后以 0 退出,因此在 harness 运行中这条警告被埋在 episode 的 stderr 里,每一轮都在默认深度下安静运行。ReasoningSpec 已把它记录为 validation="silently_ignored"——在运行开始时把这条警告在控制台显示一次,就能低成本补上这个缺口。

提议:dsh 的思考深度

DeepSeek Harness 其实有文档化的强度开关:llm-deepseek 插件接受 reasoningEffortlow/high/max)。我在之前的分支里有一个实测可行的机制:在 runner 已经写入的 --patch 覆盖层中追加一个 - id: llm-deepseek 层。已在 dsh 0.1.0-rc.7 上对真实 API 验证——dsh --dump-config 显示该值合并进插件配置,完整 MEA 运行的每个 episode 都携带它。如果你希望覆盖 dsh,我很乐意按 ReasoningSpec 注册表的架构(transport/source 字段等)提交一个小 PR,说一声即可。

另注:英语和中文都不是我的母语,此前双语发帖只是为了方便团队阅读——反正两个版本都是翻译。今后的消息您希望只用英文还是继续双语?我会照此执行。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants