Problem
A per-run metrics row carries toolCalls, startupToolCalls, numTurns and wakeupCalls. The agents[] entries inside it carry exactly:
cacheRead, cacheWrite, label, messages, usd
No tool-call count per worker. So any question of the form "did workers make fewer calls after change X" cannot be answered from metrics/runs.jsonl — it requires parsing the run's raw trace and grouping tool_use blocks by parent_tool_use_id.
Why that matters more than it looks
The raw trace is retained on a rotation; the metrics row is kept forever. So the cheap, durable artifact cannot answer the question, and the expensive, deleted one can.
Measured today: work-tokens skipped 218 of 495 metrics rows for "the trace has been collected", and only 4 producer traces with any content survived — about 21 hours of history. Every per-worker question about anything older was unanswerable, not because it was not captured, but because the capture was the half that gets deleted.
(KEEP_RUNS was 20 and is now 2000, so the immediate pressure is off. That fixes the retention; it does not change which artifact holds the answer.)
The concrete case
Producer runs either side of the non-interactive backgrounding change (#327 + thedavidmeister/claude-config#11):
| run |
poll-shaped calls |
usd |
run toolCalls |
worker mean usd |
| 20260816T170156Z (before) |
23 |
18.32 |
342 |
2.42 |
| 20260817T000148Z |
0 |
20.30 |
286 |
2.49 |
| 20260817T050103Z |
0 |
10.84 |
237 |
1.24 |
| 20260817T110404Z |
0 |
25.50 |
376 |
3.08 |
Cost cannot answer it — the post-change runs straddle the pre-change one, because run cost is dominated by what work the queue served that hour (triage vs rework), not by the change. The unit where the change acts is the worker, and the measure that isolates it from work volume is tool calls per worker, which is the one number not in the row.
Getting the "23 poll-shaped calls" figure at all required jq over parent_tool_use_id in a 20 MB trace that only still existed because the run was 21 hours old.
Proposed fix
Add toolCalls to each agents[] entry, alongside the existing messages. It is the same walk that already produces the run-level toolCalls, partitioned by parent_tool_use_id instead of totalled — the attribution key is already how agent_tokens groups spend, so nothing new has to be derived.
Consider startupToolCalls per agent too, but only if it is meaningful per worker: at run level it counts calls before firstMutationIndex, and a worker's own first mutation may not be the run's. If the per-worker analogue is not well-defined, leave it out rather than shipping a number whose meaning differs by row — one honest field beats two where one is ambiguous.
Done when
- Every
agents[] entry in a stage: "final" row carries toolCalls.
- The sum of per-agent
toolCalls plus the main loop's accounts for the run-level toolCalls, and a test pins that identity — otherwise the two can drift and nothing says so.
- A question like "tool calls per rework worker, before vs after" is answerable from
metrics/runs.jsonl alone, with no trace parsing.
Problem
A per-run metrics row carries
toolCalls,startupToolCalls,numTurnsandwakeupCalls. Theagents[]entries inside it carry exactly:No tool-call count per worker. So any question of the form "did workers make fewer calls after change X" cannot be answered from
metrics/runs.jsonl— it requires parsing the run's raw trace and groupingtool_useblocks byparent_tool_use_id.Why that matters more than it looks
The raw trace is retained on a rotation; the metrics row is kept forever. So the cheap, durable artifact cannot answer the question, and the expensive, deleted one can.
Measured today:
work-tokensskipped 218 of 495 metrics rows for "the trace has been collected", and only 4 producer traces with any content survived — about 21 hours of history. Every per-worker question about anything older was unanswerable, not because it was not captured, but because the capture was the half that gets deleted.(
KEEP_RUNSwas 20 and is now 2000, so the immediate pressure is off. That fixes the retention; it does not change which artifact holds the answer.)The concrete case
Producer runs either side of the non-interactive backgrounding change (#327 + thedavidmeister/claude-config#11):
Cost cannot answer it — the post-change runs straddle the pre-change one, because run cost is dominated by what work the queue served that hour (triage vs rework), not by the change. The unit where the change acts is the worker, and the measure that isolates it from work volume is tool calls per worker, which is the one number not in the row.
Getting the "23 poll-shaped calls" figure at all required
jqoverparent_tool_use_idin a 20 MB trace that only still existed because the run was 21 hours old.Proposed fix
Add
toolCallsto eachagents[]entry, alongside the existingmessages. It is the same walk that already produces the run-leveltoolCalls, partitioned byparent_tool_use_idinstead of totalled — the attribution key is already howagent_tokensgroups spend, so nothing new has to be derived.Consider
startupToolCallsper agent too, but only if it is meaningful per worker: at run level it counts calls beforefirstMutationIndex, and a worker's own first mutation may not be the run's. If the per-worker analogue is not well-defined, leave it out rather than shipping a number whose meaning differs by row — one honest field beats two where one is ambiguous.Done when
agents[]entry in astage: "final"row carriestoolCalls.toolCallsplus the main loop's accounts for the run-leveltoolCalls, and a test pins that identity — otherwise the two can drift and nothing says so.metrics/runs.jsonlalone, with no trace parsing.