Feature Request: Add Chat Panel with Agent Client Protocol Support

[mnm-matin]

Overview

Add a chat panel to enable natural language queries about the hyperbolic embeddings and dataset visualizations.

Proposed Implementation

Use the Agent Client Protocol (ACP): https://agentclientprotocol.com/overview/introduction

Benefits:

• Standardized protocol for agent-client communication
• Clean separation between UI and agent logic
• Support for streaming responses
• Extensible for multiple backends

Key Features

• Side panel UI for chat interface
• Query current visualization state (clusters, outliers, selected points)
• Context-aware responses based on active dataset/embeddings
• Ask about hyperbolic geometry properties and fairness metrics

[mnm-matin]

@andandandand .... the whole thing might be easier than we thought....

on second thought... doing MCP first might be a better idea

[mnm-matin]

Proposal: BYOA Curation Sidebar via ACP & MCP

I propose implementing an agentic sidebar using the Agent Client Protocol (ACP) and Model Context Protocol (MCP). This architecture supports a "Bring Your Own Agent" model, allowing users to leverage their existing subscriptions (GitHub Copilot CLI, Claude Code) to curate data directly within the app.

Architecture

The app acts as the Data Provider (via MCP) and Interface (via ACP), while the user's local CLI handles the compute and logic via a Code Sandbox.

1. Frontend (ACP Client): A sidebar UI implementing the ACP Client spec. It renders the agent's chat, plans, and state, maintaining the interactive loop.
2. Backend (The Bridge): The existing FastAPI server opens a WebSocket that spawns and bridges the user's local agent process (e.g., gh copilot --acp). It pipes stdin/stdout between the UI and the local CLI process.
3. MCP Server (Data Access): We expose generic data primitives via MCP, allowing the agent to consume data similar to how Playwright accesses page content:

• get_view_data(view_name): Returns image assets/blobs and metadata.
• get_embeddings(view_name): Returns raw vector arrays.
• update_samples(ids, changes): Generic write-back for tagging or field updates.

Example Workflow: Deduplication

• User: "Find duplicates in this view and tag them."
• Agent: Receives the prompt via ACP. It requests the view's embeddings via get_embeddings and optionally image data via get_view_data.
• Code Sandbox: The agent generates and executes Python code locally to calculate cosine similarities or cluster the vectors.
• Action: The agent calls update_samples to apply the "duplicate" tag to the identified IDs. The UI updates instantly.