Implementing Dynamic Agent Switching Based on User Intent in AutoGen

[glarunsingh]

I am new to AutoGen. We are developing a healthcare application for an insurance company with three key features: healthcare provider, claims status provider, and doctor appointment booking. For these features, we have three agents.

When a user inquires about health, the healthcare provider agent should be triggered. If, during the conversation, the user wants to book a doctor's appointment, the doctor appointment booking agent should be activated. Similarly, if the user wants to check the status of their claims, the claims status provider agent should be triggered.

I have researched AutoGen and found that there is a termination word to exit from one agent to another. However, I want to trigger the agents based on user intent dynamically. For this, I believe the dynamic group chat feature or LLM-based function call can be utilized. Unfortunately, I couldn't find any related sample codes for this.

As I am new to this, I am struggling to implement it. Could anyone share sample codes or guide me on how to approach this use case based on user intent? Additionally, within my agents, I have multiple tool or function calling codes enabled.

Here is the code I have so far:
app.py file:

import os
from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager
from dotenv import load_dotenv
import httpx
from prompt import context_prompt, health_tips_prompt, doctor_calling_prompt, claims_update_prompt
from utils.vitals import vitals_check
from utils.claims_update import claims_status
from utils.doctor_calling import doctor_booking

# Load environment variables from .env file
load_dotenv()

print(os.environ.get("GROQ_API_KEY"))

# Configure LLM settings
config_list = [{
    "model": "llama-3.3-70b-versatile",
    "api_key": os.getenv("GROQ_API_KEY"),
    "api_type": "groq",
    #"http_client": httpx.Client(verify=False)  # Debug connection
}]

# Create a healthcare assistant with prompts
health_care_agent_prompt = context_prompt + health_tips_prompt
health_care_assistant = AssistantAgent(
    name="health_assistant",
    system_message=health_care_agent_prompt,
    llm_config={
        "config_list": config_list,
        "function_call": "auto"  # Allow function calls when required
    }
)

# Create a doctor calling assistant with prompts
doctor_calling_assistant_prompt = context_prompt + doctor_calling_prompt
doctor_calling_assistant = AssistantAgent(
    name="health_assistant",
    system_message=doctor_calling_assistant_prompt,
    llm_config={
        "config_list": config_list,
        "function_call": "auto"  # Allow function calls when required
    }
)

# Create a claims assistant with prompts
claims_status_agent_prompt = context_prompt + claims_update_prompt
claims_status_assistant = AssistantAgent(
    name="health_assistant",
    system_message=claims_status_agent_prompt,
    llm_config={
        "config_list": config_list,
        "function_call": "auto"  # Allow function calls when required
    }
)

# Create a user proxy agent
user_proxy = UserProxyAgent(
    name="user_proxy",
    system_message="you will be a supervised agent, and will decide which agent to call based on user query.",
    code_execution_config=False,
    #human_input_mode="TERMINATE",
)

# Register tool functions with agents
health_care_assistant.register_for_llm(name="vitals_checkup", description="Check user's vitals")(vitals_check)
doctor_calling_assistant.register_for_llm(name='doctor_calling', description="Calls this function when user asks to book a doctor's appointment")(doctor_booking)
claims_status_assistant.register_for_llm(name='claims_status_check', description="Calls this function when user asks to check claims status")(claims_status)
user_proxy.register_for_execution(name="vitals_checkup")(vitals_check)
user_proxy.register_for_execution(name="doctor_calling")(doctor_booking)
user_proxy.register_for_execution(name="claims_status_check")(claims_status)

health_care_assistant.llm_config["tools"]

groupchat = GroupChat(agents=[user_proxy, health_care_assistant, doctor_calling_assistant, claims_status_assistant], messages=[], max_round=12)
manager = GroupChatManager(groupchat=groupchat, llm_config={"config_list": config_list})

# Start a conversation (test vitals checkup)
result = user_proxy.initiate_chat(
    health_care_assistant,
    # doctor_calling_assistant,
    # claims_status_assistant,
    message="Hi Zoe",
    summary_method="reflection_with_llm",
    # max_turns=12
)

print(result.summary)

[ekzhu]

I recommend start with the new v0.4 API, go through the tutorial, and look at SelectorGroupChat which is designed for your scenario.

Tutorial: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/index.html

SelectorGroupChat: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/selector-group-chat.html

The 0.2 API is more of a research prototype and has many flaws. Even though many YouTube videos are made for it we don't recommend using it.

[glarunsingh]

Thanks @ekzhu. I'll try it and come back if I have further queries.

[glarunsingh]

app_autogenv0.4.txt
I have written this code based on the selector group chat manager.

Issue Description:

I am encountering several challenges with the current workflow of our health care assistant system. The intended flow is as follows:

• When a user shares their symptoms, it should trigger the health_care_assistant.
• The health_care_assistant responds with a follow-up question to better understand the symptoms.
• Upon sharing the symptoms again, the system should remain in the health_care_assistant state as we are communicating with the health care assistance.
• Futher the health_care_assistant should also check for vitals using a seperate function calling.

However, the issue arises when the system incorrectly triggers the doctor_calling_agent instead of staying within the health_care_assistant. The doctor_calling_agent is designed to perform a different function, such as gathering details about the user's availability for a doctor visit and initiating a telephonic conversation with the doctor's office. This process should only occur after the health_care_assistant has checked the user's vitals, which is currently not happening.

Additionally, the system should retain all conversation history from the user and act accordingly based on the context provided.

To address this, we need to define a structure that ensures proper supervision and state management. This is similar to how we handle client requests for claim status, where we fetch information from the database and display it on the frontend.

Any assistance in resolving this issue would be greatly appreciated.

(.venv) ar20572127@FVFHXFN8Q6LR Autogen_Zoe % python3 app_autogenv0.4.py
Hi, I'm Zoe, your health care assistant. How can I help? I can assist with health checkups, doctor appointments, and claims status.
You: I have head ache
---------- user ----------
I have head ache
---------- HealthCareAgent ----------
I'm sorry to hear about your headache. Could you tell me more about it? Specifically, where is the pain located, how severe is it on a scale from 1 to 10, and how long have you been experiencing it?
You: its mild around my temples and i'm experiencing from morning
---------- user ----------
its mild around my temples and i'm experiencing from morning
---------- DoctorCallingAgent ----------
Thank you for sharing that with me. Since it's a mild headache around your temples and it's been ongoing since the morning, there are a few things we can consider to help you feel better. Ensure you're staying hydrated and try to rest in a quiet, dark room if possible. You might also consider taking a mild over-the-counter pain reliever if that's suitable for you.

If your headache persists or worsens, it might be a good idea to consult with a healthcare professional. Would you like help with anything else or more guidance on this?
You:

[glarunsingh]

I need some kind of a supervisor agent where it can actually gets the user input and based on the prompt conditions and conversation history, it should able to make the decision by calling the respective agents.

[ekzhu]

I believe this is an important pattern to address. Thanks for sharing your scenario. I believe SelectorGroupChat is not the appropriate choice for your scenario. Currently, you need to go to the Core API to create your own orchestration and agent implementation. The Core API provides an event-driven programming interface so you can implement any kind of message flows you want. There are other examples of orchestration patterns in the Core API documentation.

Let's address this in two steps:

1. Create a notebook example of showing how to use the Core API to implement the supervisor pattern you described, using the same scenario of health care.
2. Design a high-level abstraction for the AgentChat API to support the supervisor pattern.

We will need your assistance in this process. First, would you be willing to help review the PR for step 1? You will need to first read the Core API documentation and get a sense of how to use it. Then, following the notebook example, would you like to provide feedbacks when it comes to designing the high-level API for AgentChat?

It may last a few weeks minimum.

[ekzhu]

Created an issue for step 1: #5535

[glarunsingh]

Sure, I would be happy to work on this. I will go over the Core API Documentation and come back.

[glarunsingh]

@ekzhu can you share the link for this Core API please?

[ekzhu]

https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/quickstart.html