feat: LLM agents as the workflow copilot #4020
Conversation
github-actions
bot
added
feature
dependencies
|
@bobbai00 Please summarize the plan per our discussion. In the new PRs, include a architecture diagram and related screenshots. |
Xiao-zhen-Liu
left a comment
Xiao-zhen-Liu
left a comment
There was a problem hiding this comment.
Tested and works well. Left some comments mainly for improving clarity.
I also think we should have a configuration flag for this feature. It would be even better if admins can turn it on / off dynamically.
| private model: any; | ||
| private modelType = ""; | ||
| private agentName = ""; | ||
| private messages: ModelMessage[] = []; |
There was a problem hiding this comment.
Can you add some comments about these variables? Both messages and agentResponses are messages and it is not immediately clear which does what.
| try { | ||
| this.model = createOpenAI({ | ||
| baseURL: new URL(`${AppSettings.getApiEndpoint()}`, document.baseURI).toString(), | ||
| apiKey: "dummy", |
There was a problem hiding this comment.
Why can this be dummy?
| this.notificationService.info(`Agent ${this.agentName} has stopped generation`); | ||
| return true; | ||
| } | ||
| return stepCountIs(50)({ steps }); |
There was a problem hiding this comment.
What does this mean?
| const tools = this.createWorkflowTools(); | ||
| let isFirstStep = true; | ||
|
|
||
| return from( |
There was a problem hiding this comment.
This is the most import part of the life-cycle of agent handling. It would be good if you can have some docs here about what each step / callback does.
| this.emitAgentUIMessage("user", message, true, true); | ||
| this.messages.push({ role: "user", content: message }); | ||
|
|
||
| const tools = this.createWorkflowTools(); |
There was a problem hiding this comment.
Why does this have to be created inside sendMessage? Can it be created once inside this class?
| <li | ||
| nz-menu-item | ||
| *ngIf="hasHighlightedLinks() && | ||
| *ngIf="hasHighlightedLinks() && |
| return this.agentState === CopilotState.GENERATING; | ||
| } | ||
|
|
||
| public isStopping(): boolean { |
There was a problem hiding this comment.
Where is this method used?
|
|
||
| ngOnInit(): void { | ||
| // Load saved panel dimensions and position | ||
| const savedWidth = localStorage.getItem("agent-panel-width"); |
There was a problem hiding this comment.
It would be clearer if the logics for setting / loading local storage are contained in their own methods.
| onResize({ width, height }: NzResizeEvent): void { | ||
| cancelAnimationFrame(this.id); | ||
| this.id = requestAnimationFrame(() => { | ||
| this.width = width!; |
There was a problem hiding this comment.
What would happen if width or height are undefined? Should there be some default values for such case?
| /** | ||
| * Create a new agent with the selected model type. | ||
| */ | ||
| public async createAgent(): Promise<void> { |
There was a problem hiding this comment.
This method is marked async but doesn’t use await or return a Promise-based flow. Since the logic uses RxJS and subscribe(), it doesn’t need to be async.
3 participants
What changes were proposed in this PR?
This PR introduces the LLM agent management & chat panel on the workflow workspace to help users with their workflows.
Demo
Manage agent using the panel
Ask agent questions regarding available Texera operators
Ask agent about users' current workflow
Architecture Diagram
See #4034
Major Changes
Frontend: introduce the agent management & chat panel
Backend:
litellmis introduced: which is a open source service that manages the communication between app and LLM APIsAccessControlServiceis modified: adding the logic for routinglitellmrelated requestsAny related issues, documentation, discussions?
Related to #4034
Current PR limitation and future PR plans
In current PR, the agent is only able to act in a "read-only" way, meaning it can only answer questions regarding operators, but couldn't change user's workflow.
In future PRs,
How was this PR tested?
Frontend unit test cases are added.
To test the PR e2e:
bin/litellm-config.yamlAccessControlServiceWas this PR authored or co-authored using generative AI tooling?
The code content is co-authored with Claude code. This PR is not generated by generative AI.