feat: chat conversation

[hanakannzashi]

Completion for #120

[gemini-code-assist]

Summary of Changes

Hello @hanakannzashi, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the interactive capabilities of the application by integrating real-time chat features through WebSockets. It establishes a robust framework for managing WebSocket connections and broadcasting various conversation-related events, such as typing indicators and new message creations, to connected clients. This foundational work paves the way for a more dynamic and responsive user experience in conversations.

Highlights

• WebSocket Integration: Introduced comprehensive WebSocket support to enable real-time communication for chat conversations.
• Connection Management: Implemented a ConnectionManager to efficiently handle and broadcast messages to WebSocket subscribers for specific conversations.
• Typing Indicator API: Added a new API endpoint /v1/conversations/{conversation_id}/typing that allows users to broadcast typing indicators to other participants in a conversation.
• Real-time Conversation Updates: Enhanced the AuthorTrackingStream to broadcast response_created events via WebSockets once a new conversation item is fully generated, providing immediate feedback to clients.
• Dependency Updates: Updated Cargo.toml and Cargo.lock to include necessary dependencies such as tokio-tungstenite, futures-util, base64, and sha1 to support WebSocket functionality.

Changelog

• Cargo.lock
  • Added futures-util dependency.
  • Added tokio-tungstenite dependency (versions 0.21.0 and 0.28.0).
  • Added base64 dependency.
  • Added sha1 dependency.
  • Added tungstenite dependency (versions 0.21.0 and 0.28.0).
  • Added utf-8 dependency.
• crates/api/Cargo.toml
  • Enabled the ws feature for the axum dependency.
  • Added tokio-tungstenite (version 0.21) as a dependency.
  • Added futures-util (version 0.3) as a dependency.
• crates/api/src/lib.rs
  • Exported the new websocket module.
  • Exported ConnectionManager and WebSocketMessage from the websocket module.
• crates/api/src/main.rs
  • Imported ConnectionManager.
  • Initialized ConnectionManager and added it to the AppState.
• crates/api/src/openapi.rs
  • Added the typing_indicator route to the OpenAPI documentation.
• crates/api/src/routes/api.rs
  • Added a new POST route /v1/conversations/{conversation_id}/typing for the typing_indicator handler.
  • Modified create_conversation_items to broadcast new items to WebSocket subscribers.
  • Implemented the typing_indicator function to broadcast typing events.
  • Modified proxy_responses to pass the ConnectionManager to AuthorTrackingStream.
  • Updated AuthorTrackingStream struct to include response_id and connection_manager.
  • Modified AuthorTrackingStream::new to accept connection_manager.
  • Updated AuthorTrackingStream to store response_id and broadcast response_created events when the stream finishes.
• crates/api/src/routes/mod.rs
  • Cloned auth_state for the API routes.
  • Added WebSocket routes for /v1/ws/conversations/{conversation_id}.
  • Merged WebSocket routes into the main router.
• crates/api/src/state.rs
  • Imported ConnectionManager.
  • Added connection_manager (Arc) to AppState.
• crates/api/tests/common.rs
  • Imported ConnectionManager.
  • Initialized ConnectionManager for test server setup.
  • Added connection_manager to the AppState in test server creation.

Activity

• The pull request introduces a new websocket module to encapsulate WebSocket-related logic.
• Core application state (AppState) has been extended to manage WebSocket connections centrally.
• New dependencies for WebSocket handling and related utilities have been added to the project.
• API routing has been updated to include a dedicated WebSocket endpoint and a new REST endpoint for typing indicators.
• Existing data streaming logic has been refactored to integrate real-time event broadcasting.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[claude]

⚠️ CRITICAL ISSUES - Cannot Merge

This PR has blocking compilation errors and several critical issues that must be fixed before merge.

---

🔴 BLOCKING: Missing Module File

Issue: The PR declares pub mod websocket; in lib.rs but the websocket.rs file does not exist.

Impact: This PR will fail to compile. The code references:

• crate::websocket::ConnectionManager
• crate::websocket::WebSocketMessage
• crate::websocket::websocket_handler

But none of these are defined anywhere in the PR.

Fix: Add the missing crates/api/src/websocket.rs file to this PR.

---

🔴 CRITICAL: Privacy Violation in Logging

// crates/api/src/routes/api.rs:1184-1190
tracing::debug!(
    "Broadcast new items to {} subscribers for conversation_id={}",
    subscriber_count,
    conversation_id
);

Issue: Per CLAUDE.md, logging at info level and above runs in production. While this uses debug!, the pattern is risky.

Concern: If response_json is ever logged (not currently, but easy mistake), it would expose customer conversation content.

Recommendation:

• Keep logging at debug level only ✓
• Never log response_json content (add comment warning about this)
• Ensure all WebSocket broadcast logging follows the same pattern

---

🟡 IMPORTANT: Missing Error Handling

Issue: WebSocket broadcasts in streaming responses lack timeout/error handling:

// crates/api/src/routes/api.rs:3277-3293
tokio::spawn(async move {
    let subscriber_count = connection_manager
        .broadcast_response_created(&conv_id, response_id.as_deref())
        .await;
    // ...
});

Concerns:

1. No timeout on the spawn - if broadcast_response_created hangs, it leaks the task
2. No error handling if broadcast fails
3. Spawned task could outlive the stream without cleanup

Recommendation:

tokio::spawn(async move {
    let timeout_duration = Duration::from_secs(5);
    match tokio::time::timeout(timeout_duration, connection_manager.broadcast_response_created(&conv_id, response_id.as_deref())).await {
        Ok(subscriber_count) => {
            if subscriber_count > 0 {
                tracing::debug!("Broadcast response_created to {} subscribers for conversation_id={}", subscriber_count, conv_id);
            }
        }
        Err(_) => {
            tracing::warn!("WebSocket broadcast timeout for conversation_id={}", conv_id);
        }
    }
});

---

🟡 Code Quality: Resource Management

Issue: ConnectionManager is added to AppState but we can't review:

• WebSocket connection cleanup on client disconnect
• Memory leak prevention for orphaned connections
• Graceful shutdown handling

Action Required: When adding websocket.rs, ensure:

1. Connections are removed from the manager on disconnect
2. Broadcast methods handle closed connections gracefully
3. ConnectionManager::new() doesn't spawn background tasks that outlive the app

---

🔵 Minor: Dependency Versions

Two versions of tokio-tungstenite in dependency tree:

• 0.21.0 (in dev-dependencies)
• 0.28.0 (via axum)

Impact: Low - different contexts, but creates confusion.

Recommendation: Align to 0.28 if possible for consistency.

---

Summary

Cannot merge until:

1. ✅ Add missing crates/api/src/websocket.rs file
2. ✅ Add timeout + error handling to spawned broadcast tasks
3. ✅ Verify WebSocket connection cleanup in ConnectionManager

Sign-off: ⚠️ BLOCKING ISSUES FOUND

---

_{Review performed by Claude Code following repository CLAUDE.md guidelines}

[gemini-code-assist]

Code Review

This pull request introduces real-time conversation features using WebSockets, including broadcasting new messages, typing indicators, and response creation events. The implementation adds a new WebSocket route, a ConnectionManager to handle connections, and integrates broadcasting logic into existing API endpoints. The changes are generally well-structured. I have a few suggestions regarding dependency management, performance, and documentation accuracy to enhance the code quality.

[gemini-code-assist]

There's a version mismatch for tokio-tungstenite. The ws feature of axum (v0.8) brings in tokio-tungstenite v0.28.0, but the dev-dependencies specify v0.21. Using different versions for testing and production can lead to subtle bugs due to API differences. It's recommended to align the dev dependency with the version used by axum to ensure consistency.

Suggested change

	tokio-tungstenite = "0.21"
	tokio-tungstenite = "0.28"

[gemini-code-assist]

Cloning response_json (a serde_json::Value) can be inefficient, especially if the conversation item is large. To improve performance, consider modifying broadcast_new_items to accept a reference (&serde_json::Value) instead of taking ownership. This avoids a potentially expensive clone at the call site. Inside broadcast_new_items, you could then serialize the value once and distribute cheap clones of the serialized Bytes to each WebSocket connection.

Suggested change

	.broadcast_new_items(&conversation_id, response_json.clone())
	.broadcast_new_items(&conversation_id, &response_json)

[gemini-code-assist]

The documentation for the typing_indicator endpoint states that it requires "at least read access", but the implementation correctly checks for SharePermission::Write. The documentation should be updated to reflect the actual requirement to avoid confusion for API consumers. Write access is more appropriate for this action.

Suggested change

	/// Requires authentication and at least read access to the conversation.
	/// Requires authentication and at least write access to the conversation.