feat(diff): rich side-by-side session comparison - #52
Merged
Merged
Conversation
Add --compare flag to agent-strace diff. Produces a structured table comparing two sessions across cost, duration, tool calls, redundant reads, context resets, files modified, and errors, with a deterministic verdict. agent-strace diff <session-a> <session-b> --compare New metrics: - Redundant reads: files read more than once in a session - Context resets: LLM requests separated by >120s gap - Approach divergence: first N phase pairs where behaviour differs - Verdict: derived from cost/duration/error comparisons (no LLM) Closes #47 Co-authored-by: Ona <no-reply@ona.com>
- format_compare: Duration and Cost rows now show a % change column. Previously _row received pre-formatted strings so _pct() never ran; now raw numeric values are passed separately for the change calculation. - format_diff: when sessions share no aligned phases (divergence_index=0 and phase_diffs is empty), print 'diverge from the start' instead of 'Diverged at phase 1' which implied a specific phase existed. - compare_sessions _verdict: remove dead lower_better variable; guard division by zero when vb==0 was already present but comment clarified. Co-authored-by: Ona <no-reply@ona.com>
Co-authored-by: Ona <no-reply@ona.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #47
What
Adds
--comparetoagent-strace diff— a structured side-by-side table comparing two sessions with a deterministic verdict.Usage
Changes
src/agent_trace/diff.pyCompareReportdataclass: holds all comparison metricscompare_sessions(): computes redundant reads, context resets, files modified, approach divergence, and verdictformat_compare(): renders the table_count_redundant_reads(): counts files read more than once per session_count_context_resets(): counts LLM request gaps >120s_build_label(): derives a short display label from session metadatacmd_diff(): routes tocompare_sessionswhen--compareis setsrc/agent_trace/cli.py--compareflag ondiffsubcommandtests/test_diff_compare.py: 6 tests covering report structure, formatting, redundant read counting, context reset counting, and CLI flag.