Share LLM conversation context between chatbot/workflow runtimes during debug via conversation ID mapping#334
Open
forhad-hosain wants to merge 3 commits intodevfrom
Open
Share LLM conversation context between chatbot/workflow runtimes during debug via conversation ID mapping#334forhad-hosain wants to merge 3 commits intodevfrom
forhad-hosain wants to merge 3 commits intodevfrom
Conversation
Allow chatbot and workflow runtimes to share LLM conversation context during debug sessions. The chatbot runtime stores a debug-to-conversation mapping in cache, which the workflow runtime retrieves to use the same LLM cache. This enables the GenAILLM "Use Context Window" feature to maintain conversation history across runtime boundaries. Key changes: - AgentRuntime: init LLM cache with conversation-based cache IDs, store debug session mappings for cross-runtime retrieval - LLMCache: add static generateCacheId for consistent cache key format - LLMContext: accept optional conversationId for cache key generation - Conversation/Chat: propagate conversationId through the stack Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Description
Problem: When a GenAILLM component has Use Context Window enabled, the Chatbot debugger fails to access the context. This happens because the Chatbot and the Debugger run on different instances, causing the context to be lost.
Solution: We now link the conversationId with a debugId. LLM messages are stored using the conversationId. When a request comes from the debugger, we retrieve the missing conversationId from the cache to correctly load the message history.
🔗 Clickup Ticket
🔧 Type of Change
✅ Checklist