Skip to content

feat(llm): Multi-LLM Provider Support (#105) #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sanggggg merged 5 commits into from
Jan 2, 2026
Merged

feat(llm): Multi-LLM Provider Support (#105) #109

sanggggg merged 5 commits into from
Jan 2, 2026

Conversation

@sanggggg
Copy link
Collaborator

@sanggggg sanggggg commented Jan 2, 2026

Summary

Add an LLM abstraction layer enabling RetroChat's analysis and summarization features to use multiple providers:

  • Google AI API (existing) - Remote API calls
  • Claude Code CLI (new) - Local subprocess via claude -p
  • Gemini CLI (new) - Local subprocess via gemini

Changes

New Module: services/llm/

  • traits.rs: LlmClient async trait for provider abstraction
  • types.rs: LlmProvider enum, GenerateRequest, GenerateResponse, LlmConfig
  • errors.rs: LlmError with is_retryable(), retry_after_secs(), user_message()
  • subprocess.rs: CLI execution utilities with timeout support
  • factory.rs: LlmClientFactory for provider selection
  • adapters/: GoogleAiAdapter, ClaudeCodeClient, GeminiCliClient

CLI Integration

  • Add --provider (-P) and --model (-m) flags to:
    • analysis run
    • summarize turns
    • summarize sessions
  • Support environment variables: RETROCHAT_LLM_PROVIDER, RETROCHAT_LLM_MODEL

Provider Isolation (Claude Code)

  • --no-session-persistence: Prevents saving to .claude/projects/ (avoids polluting retrochat imports)
  • --setting-sources user: Skips project/local CLAUDE.md files (consistent behavior)

Usage

# Google AI (default, requires GOOGLE_AI_API_KEY)
cargo cli -- analysis run SESSION_ID

# Claude Code CLI
cargo cli -- analysis run SESSION_ID --provider claude-code

# Gemini CLI
cargo cli -- summarize turns --all --provider gemini-cli

# With custom model
cargo cli -- analysis run SESSION_ID -P claude-code -m claude-sonnet-4-20250514

# Via environment variable
RETROCHAT_LLM_PROVIDER=claude-code cargo cli -- analysis run SESSION_ID

Known Limitations

  • Gemini CLI lacks --no-session-persistence flag (documented with TODO)
  • Rubric evaluation runs sequentially with CLI providers (trait object limitation)

Test Plan

  • Unit tests for all new modules (242 tests pass)
  • cargo clippy --workspace -- -D warnings passes
  • cargo fmt --check passes
  • Manual test with each provider

Closes #105

🤖 Generated with Claude Code

sanggggg and others added 5 commits January 2, 2026 15:34
@sanggggg @claude
feat(llm): add multi-provider LLM abstraction layer (#105)
69457b6 
Add support for multiple LLM providers (Google AI API, Claude Code CLI,
Gemini CLI) via a unified trait-based abstraction.

New module structure:
- llm/types.rs: LlmProvider, GenerateRequest, GenerateResponse, LlmConfig
- llm/traits.rs: LlmClient async trait
- llm/errors.rs: LlmError with retry logic
- llm/subprocess.rs: CLI execution utilities
- llm/factory.rs: LlmClientFactory for provider selection
- llm/adapters/: GoogleAiAdapter, ClaudeCodeClient, GeminiCliClient

CLI changes:
- Add --provider (-P) and --model (-m) flags to analysis run command
- Support RETROCHAT_LLM_PROVIDER and RETROCHAT_LLM_MODEL env vars

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@sanggggg
Merge remote-tracking branch 'origin/main' into vendotr
57279fb
@sanggggg @claude
feat(summarize): integrate multi-LLM provider support
9f89771 
Update TurnSummarizer and SessionSummarizer to use Arc<dyn LlmClient>
instead of GoogleAiClient, enabling use of any configured LLM provider.

Changes:
- Update TurnSummarizer to accept Arc<dyn LlmClient>
- Update SessionSummarizer to accept Arc<dyn LlmClient>
- Add --provider and --model flags to summarize turns/sessions commands
- Use dynamic model name from LlmClient instead of hardcoded value

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@sanggggg @claude
fix(llm): prevent Claude Code CLI from polluting session history
366e6cf 
Add flags to Claude Code CLI invocations to ensure clean, isolated
execution:

- --no-session-persistence: Don't save to .claude/projects/ (avoids
  polluting retrochat imports)
- --setting-sources user: Only load user settings, skip project/local
  CLAUDE.md files (avoids unexpected behavior from repo configs)

Also add TODO comment for Gemini CLI which lacks these isolation flags.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@sanggggg @claude
refactor(llm): use stdin for CLI prompts instead of args
2b9d7d6 
Pass prompts via stdin instead of command-line arguments to:
- Avoid OS argument length limits (typically 128KB-2MB)
- Handle special characters without escaping issues
- Support arbitrarily large prompts

Add run_cli_command_with_stdin() utility and update both
Claude Code and Gemini CLI adapters to use it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@sanggggg sanggggg merged commit bbe54a3 into main Jan 2, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Multi-LLM Provider Support: Claude Code & Gemini CLI Headless Mode

2 participants

@sanggggg