Research: LLM-powered PR Walkthrough / Chain-of-Thought Review UI

[tanvesh01]

Research: LLM-powered PR Walkthrough / Chain-of-Thought Review UI

Context: We want a structured, step-by-step PR walkthrough in Rudu — similar to Codiff's "Walkthrough" feature — displayed as a vertical Chain-of-Thought timeline with collapsible steps, clickable file chips, and progress tracking.

This issue catalogs what already exists in open PRs, what comparable tools do, and what infrastructure we still need.

---

1. What already exists in Rudu

PR #56 — [codex] Add remote Pi review workflow (branch: pr-worker-codex-review, 63 files)

Backend (Rust + Tauri):

• ReviewSession / ReviewWorkspace / ReviewChat domain model with persistence (session.json)
• Local git worktree checkout per PR (~/rudu/workspaces/<repo>/pr-<n>/repo)
• ACP (Agent Client Protocol) runtime adapter (review_session/acp.rs) for Codex chat
• Streaming event model: ReviewChatEvent union with Message, Thought, Tool, Plan, Finished, Error
• ReviewWorkspaceEvent log stream for workspace preparation steps
• Research ADRs document the Codex migration from Pi (0003-use-codex-as-the-review-chat-agent.md)

Frontend (React + Tailwind):

• src/components/ai-elements/chat.tsx — reusable UI primitives already built:
  • <Reasoning> — collapsible reasoning block with streaming state
  • <Tool> — tool call status block (input-available / output-available)
  • <Checkpoint> / <CheckpointIcon> / <CheckpointTrigger> — timeline markers
  • <Conversation> / <ConversationContent> — scrollable chat container
• src/features/review-chat/assistant-part.tsx — renders assistant message parts:
  • text → markdown
  • reasoning → <Reasoning>
  • data-acp-plan → AcpPlanView (ordered plan entries with status badges)
  • tool → <Tool>
• src/features/review-chat/line-selection.ts — file attachments with diff line ranges, already wired into PatchViewerMain
• Chat panel added as a right sidebar tab ("Rudu") in patch-viewer-main.tsx

Key insight: The UI primitives for a Chain-of-Thought timeline are already 70% built. They are currently arranged for a chat transcript, not a walkthrough.

PR #29 — Add AI summarization review panels (branch: d4rm5:main, 24 files)

Backend:

• src-tauri/src/services/chapters.rs — LLM chapter generator with structured JSON prompt (chapters-v1)
• Prompt returns: { prologue: { summary, keyChanges[], reviewFocus[] }, chapters: [{ title, summary, files[], reviewSteps[], risks[] }] }
• 8-provider LLM abstraction in services/llm.rs (OpenAI, Anthropic, Google, OpenRouter, Z.ai, Minimax, Opencode, OpenAI-compatible)
• Keyring-based API key storage (rudu.llm)
• SQLite caching for chapters

Frontend:

• src/components/ui/chapter-overview.tsx — ~1000-line component rendering chapter list with severity badges, expandable chapters, review steps, related files, risks panel, completion tracking
• TanStack Query hooks in src/queries/llm.ts

Key insight: PR #29 has a structured backend for generating grouped review content, but the UI is a dashboard, not a walkthrough timeline. The data shape is close to what a walkthrough needs.

---

2. What Codiff does (reference implementation)

Codiff's walkthrough (electron/walkthrough.cjs):

• Prompt strategy: Returns a review walkthrough order, not review findings. Files are ranked by "review leverage" (architecture boundaries → contained changes → mechanical churn).
• JSON schema:

  {
    "groups": [
      {
        "title": "Review carefully",
        "reason": "...",
        "files": [
          {
            "path": "...",
            "action": "review" | "scan" | "skim",
            "impact": "wide" | "contained" | "mechanical",
            "reason": "...",
            "context": "..."
          }
        ]
      }
    ],
    "summary": { "focus": "...", "skim": "..." }
  }

• Normalization: Validates every file is used exactly once; fills missing files into a fallback group.
• Model: gpt-5.3-codex-spark with reasoning effort high, 45s timeout, read-only sandbox.

---

3. What ai-sdk ChainOfThought offers (Vercel AI Elements)

Composable component API (packages/elements/src/chain-of-thought.tsx):

<ChainOfThought defaultOpen>
  <ChainOfThoughtHeader />
  <ChainOfThoughtContent>
    <ChainOfThoughtStep
      icon={SearchIcon}
      label="Searching for profiles..."
      status="complete" | "active" | "pending"
    >
      <ChainOfThoughtSearchResults>
        <ChainOfThoughtSearchResult>github.com</ChainOfThoughtSearchResult>
      </ChainOfThoughtSearchResults>
    </ChainOfThoughtStep>
  </ChainOfThoughtContent>
</ChainOfThought>

• Vertical step list with connector line
• status prop: complete | active | pending
• Collapsible header
• Search result chips (badges)
• Image support

Mapping to PR walkthrough:

ChainOfThought concept	PR walkthrough mapping
<ChainOfThoughtStep>	One file group (e.g. "Review carefully")
label	Group title + reason
status	User progress through groups
<ChainOfThoughtSearchResult>	File chip with action/impact badges
description	File reason + context
Click on chip	Scroll to file in diff viewer

---

4. Comparable tools in the wild

Tool	Stars	Walkthrough?	Structured output?	Notes
PR-Agent	11.2k	Partial (/describe, /review)	Yes (YAML/JSON configs)	Mature, but GitHub-action-centric; no desktop timeline UI
OpenCommit	7.1k	No	No	Commit message generation only
slop-review	~50	Yes (CLI)	Yes (JSON)	Lightweight Ruby CLI; similar "impact" concept
diff-explainer	~200	No	No	Explains hunks in plain English
wrily	~100	Yes	Yes	Web-based review assistant with step-by-step guidance

None of the open-source tools provide a native desktop Chain-of-Thought timeline UI. Rudu would be novel here.

---

5. Gap analysis

Need	PR #56 has	PR #29 has	Missing
Local workspace for LLM context	✅	❌	—
Streaming event model	✅	❌	—
Reasoning/tool/status UI primitives	✅	❌	—
Structured grouped output backend	❌	✅	—
impact / action per file	❌	❌	✅ NEW
Vertical timeline layout	❌	❌	✅ NEW
Step status (complete/active/pending)	Partial (tool states)	❌	✅ NEW
File chip → diff scroll	Partial (attachments)	❌	✅ NEW
Summary header (focus + skim)	❌	Partial (prologue)	✅ Extend

---

6. Recommended implementation paths

Option A: Extend PR #56 chat (chat-driven walkthrough)

Add a special /walkthrough prompt that instructs Codex to output a structured walkthrough using the existing ACP plan format, enriched with file groups. Render it by restyling the existing <Reasoning> + <Tool> + <Checkpoint> stack into a vertical timeline.

Pros:

• No new backend command needed
• Leverages existing Codex ACP session
• Interactive: user can ask follow-up questions about each step

Cons:

• Relies on Codex reliably producing structured JSON inside a chat turn
• Harder to cache/regenerate
• Tied to Codex (can't easily swap providers)

Option B: Add separate walkthrough backend (pre-generated)

New Tauri command + prompt (inspired by PR #29's chapters.rs but simpler), returning the Codiff JSON shape. New React component renders it as a Chain-of-Thought timeline. Reuse file chip click handler from PR #56's attachment system.

Pros:

• Deterministic structured output with JSON schema enforcement
• Provider-agnostic (uses PR Add AI summarization review panels #29's 8-provider LLM service)
• Cacheable per PR revision
• Matches Codiff's proven prompt strategy

Cons:

• New Rust module + command
• New frontend component (though 70% of primitives exist)
• Needs merge/rebase of PR Add AI summarization review panels #29's LLM infrastructure

Option C: Hybrid (recommended)

Build a dedicated walkthrough backend (Option B) for the initial structured generation, but render it inside the Review Chat panel (Option A) so the user can ask follow-up questions about any step. The walkthrough appears as the first assistant message in the chat, using the Chain-of-Thought layout.

Why this is best:

• Gets the reliability of structured JSON generation
• Keeps the user inside the conversational review flow
• Reuses PR codex acp #56's chat UI primitives
• Can fall back to "just chat" if walkthrough generation fails

---

7. Concrete next steps (if we proceed)

1. Branch off PR codex acp #56 (or wait for merge) — the chat infrastructure is the right foundation.
2. Port PR Add AI summarization review panels #29's LLM service (services/llm.rs, commands/chapters.rs, types/github.ts) into the same branch.
3. Add impact and action fields to the chapter/walkthrough prompt and TypeScript types.
4. Create WalkthroughTimeline component in src/features/review-chat/ using existing <Reasoning> / <Tool> / <Checkpoint> primitives, styled as a vertical timeline.
5. Wire file chips to useDiffNavigator scroll-to-file (already exists in patch-viewer-main.tsx).
6. Add per-step status tracking based on which files the user has viewed (could use existing diff navigator state).
7. Cache walkthroughs per PR + head SHA using the existing SQLite cache or session metadata.

---

8. Open questions

• Should the walkthrough be auto-generated on PR open or on-demand via a button?
• Should we support re-running the walkthrough with a different provider/model?
• How do we handle large PRs (>120k chars patch)? PR Add AI summarization review panels #29 already clips; Codiff uses 160k total with 4k per section.
• Do we want user-editable plan entries (like Codex's checkpoint interaction) or read-only?
• Should file chips show only action (review/scan/skim) or also impact (wide/contained/mechanical)?

---

Research compiled from:

• PR codex acp #56 (pr-worker-codex-review) — Codex review session with ACP
• PR Add AI summarization review panels #29 (d4rm5:main) — AI summarization / chapters
• nkzw-tech/codiff — electron/walkthrough.cjs
• vercel/ai-elements — packages/elements/src/chain-of-thought.tsx
• The-PR-Agent/pr-agent and related tools

[tanvesh01]

This was generated by AI during triage.

Agent Brief (Ready-for-Human)

Goal: Decide whether and how to implement an LLM-powered PR Walkthrough / Chain-of-Thought Review UI in Rudu.

Why this needs human decision:

• The issue catalogs three competing implementation paths (chat-driven walkthrough, pre-generated backend, hybrid) without a mandated choice.
• Open product questions remain: auto-generation vs on-demand, model/provider swapping, large PR clipping strategy, read-only vs editable plan entries.
• The feature intersects with two unmerged PRs (codex acp #56 Codex review session, Add AI summarization review panels #29 AI summarization panels) and requires sequencing decisions.
• The scope spans new Rust backend commands, new React components, and prompt engineering — a cross-stack design decision.

Current codebase context:

• PR codex acp #56 (unmerged) provides ACP chat infrastructure, streaming events, and UI primitives (, , ) already ~70% built.
• PR Add AI summarization review panels #29 (unmerged) provides structured chapter backend and 8-provider LLM service.
• Main branch does not yet contain either PR's changes.
• The Codiff walkthrough JSON schema and Vercel AI SDK ChainOfThought components are referenced as design targets.

Acceptance criteria (pending human direction):

• Choose Option A, B, or C (or reject).
• If proceeding, split the chosen path into focused, agent-ready issues (backend prompt+command, frontend timeline component, file-chip navigation, status tracking).
• Resolve open questions: auto-generation vs on-demand, provider/model re-run support, large PR handling, user-editable plans, badge scope.

Out of scope for this issue:

• Direct agent implementation from this meta-issue.
• Maintaining this as a permanent living research document rather than a decision forcing function.

[tanvesh01]

This was generated by AI during repo maintenance.

📋 Partial implementation found on main

Already implemented:

• Dedicated walkthrough backend exists: src-tauri/src/services/review_session/walkthrough.rs validates the session, writes a JSON schema, asks Codex, parses ReviewWalkthrough, and normalizes changed files (lines 89-139).
• Structured walkthrough shape exists: src-tauri/src/models/mod.rs defines ReviewWalkthrough, groups, files, ReviewWalkthroughAction, and ReviewWalkthroughScope (lines 348-392).
• On-demand generation exists: src/features/review-chat/onboarding/empty-state.tsx renders a Review walkthrough button when generation is available (lines 17-26).
• Chat-panel rendering exists: src/features/review-chat/walkthrough/view/review-walkthrough-view.tsx renders summary focus/skim text and ordered group cards (lines 16-40).
• File chip selection is wired: src/features/review-chat/walkthrough/view/file-row.tsx calls onSelectFile(file.path) from the file badge button (lines 22-30).

Still missing or intentionally unresolved:

• Provider/model swapping and re-running with a different provider is not present; the generator currently calls Codex (run_codex_exec) directly.
• User-editable plan entries/status tracking are not present in the walkthrough view.
• The issue's original ready-for-human decision work (whether to choose Option A/B/C or split follow-up issues) is only partly superseded by the current implementation.

Leaving open for human review of the remaining product decisions and any follow-up issue splitting.

[tanvesh01]

This was generated by AI during repo maintenance.

📋 Partial implementation update on main

Newly verified since the last audit:

• Walkthrough generation now follows the selected review runtime instead of Codex-only dispatch:
  src-tauri/src/services/review_session/walkthrough_generator/mod.rs routes ReviewChatRuntimeKind::Codex to
  codex::run and ReviewChatRuntimeKind::OpenCode to opencode::run (lines 16-20).
• Runtime-specific progress text exists in the same module (lines 23-28), so the UI can distinguish Codex vs OpenCode
  walkthrough generation.
• OpenCode has a dedicated walkthrough generator implementation at
  src-tauri/src/services/review_session/walkthrough_generator/opencode.rs, including JSON parse/repair handling.

Still missing or intentionally unresolved:

• User-editable plan entries/status tracking are still not present in the walkthrough view.
• The issue's original ready-for-human decision work is still only partly superseded: follow-up scope still needs a human
  decision/splitting if the current on-demand walkthrough should become a fuller Chain-of-Thought/review-plan workflow.

Leaving open for human review of the remaining product decisions.

[tanvesh01]

This was generated by AI during repo maintenance.

✅ Verified: this is implemented on main

Evidence:

• Chosen implementation path is documented in src/features/review-chat/docs/adr/0007-generate-review-walkthroughs-with-codex-exec-schema.md (lines 3-7): v1 walkthroughs use a sidecar generator and are displayed inside Review Chat.
• The backend command path exists in src-tauri/src/services/review_session.rs (lines 524-568): generate_review_walkthrough / run_review_walkthrough_turn prepare the workspace and run walkthrough generation.
• The walkthrough prompt/schema exists in src-tauri/src/services/review_session/walkthrough.rs (lines 100-126): it requires JSON, review order, summary/grouped files, action, scope, reason, and context.
• The frontend command is wired in src/features/review-chat/panel/use-walkthrough-command.ts (lines 87-163): it sends /walkthrough, tracks progress, and calls runReviewWalkthroughTurn.
• The UI renders the walkthrough in Review Chat at src/features/review-chat/walkthrough/view/review-walkthrough-view.tsx (lines 16-40), with collapsible group cards in group-card.tsx (lines 21-71).
• File chips navigate back to the diff via onSelectFile in src/features/review-chat/walkthrough/view/file-row.tsx (lines 22-30).
• The shared TypeScript data shape is present in src/types/github.ts (lines 184-208): ReviewWalkthroughAction, ReviewWalkthroughScope, groups, files, and summary.

Closing as completed.