AGS Improvements (Add Test Button in Team Builder View + Others) (#5416)

## Why are these changes needed? - Add ability to test teams in Team Builder view - Update Gallery (add deep research default team, fix bug with gallery serialization) - UI fixes - improve drag drop component experience - improve new session experience (single click rather than 3 clicks to create a session) - fix bug with stop reason not being shown in some cases <img width="1738" alt="Image" src="https://github.com/user-attachments/assets/4b895df2-3bad-474e-bec6-4fbcbf1c4346" /> <img width="1761" alt="Image" src="https://github.com/user-attachments/assets/65f52eb9-e926-4168-88fb-d2496c159474" />  ## Related issue number Closes #5392  ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
microsoft · Feb 7, 2025 · 9494ac9 · 9494ac9
1 parent 73a7ba5
commit 9494ac9
Show file tree

Hide file tree

Showing 20 changed files with 668 additions and 141 deletions.
diff --git a/python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/faq.md b/python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/faq.md
@@ -15,51 +15,95 @@ A: You can specify the directory where files are stored by setting the `--appdir
 
 Yes. AutoGen standardizes on the openai model api format, and you can use any api server that offers an openai compliant endpoint.
 
-AutoGen Studio is based on declaritive specifications which applies to models as well. Agents can include a model_client field which specifies the model endpoint details including `model`, `api_key`, `base_url`, `model type`.
+AutoGen Studio is based on declaritive specifications which applies to models as well. Agents can include a model_client field which specifies the model endpoint details including `model`, `api_key`, `base_url`, `model type`. Note, you can define your [model client](https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/components/model-clients.html) in python and dump it to a json file for use in AutoGen Studio.
 
-An example of the openai model client is shown below:
+In the following sample, we will define an OpenAI, AzureOpenAI and a local model client in python and dump them to a json file.
+
+```python
+from autogen_ext.models.openai import AzureOpenAIChatCompletionClient, OpenAIChatCompletionClient
+from autogen_core.models import ModelInfo
+
+model_client=OpenAIChatCompletionClient(
+            model="gpt-4o-mini",
+        )
+print(model_client.dump_component().model_dump_json())
+
+az_model_client = AzureOpenAIChatCompletionClient(
+    azure_deployment="{your-azure-deployment}",
+    model="gpt-4o",
+    api_version="2024-06-01",
+    azure_endpoint="https://{your-custom-endpoint}.openai.azure.com/",
+    api_key="sk-...",
+)
+print(az_model_client.dump_component().model_dump_json())
+
+mistral_vllm_model = OpenAIChatCompletionClient(
+        model="TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
+        base_url="http://localhost:1234/v1",
+        model_info=ModelInfo(vision=False, function_calling=True, json_output=False, family="unknown"),
+    )
+print(mistral_vllm_model.dump_component().model_dump_json())
+```
+
+OpenAI
 
 ```json
 {
-  "model": "gpt-4o-mini",
-  "model_type": "OpenAIChatCompletionClient",
-  "api_key": "your-api-key"
+  "provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
+  "component_type": "model",
+  "version": 1,
+  "component_version": 1,
+  "description": "Chat completion client for OpenAI hosted models.",
+  "label": "OpenAIChatCompletionClient",
+  "config": { "model": "gpt-4o-mini" }
 }
 ```
 
-An example of the azure openai model client is shown below:
+Azure OpenAI
 
 ```json
 {
-  "model": "gpt-4o-mini",
-  "model_type": "AzureOpenAIChatCompletionClient",
-  "azure_deployment": "gpt-4o-mini",
-  "api_version": "2024-02-15-preview",
-  "azure_endpoint": "https://your-endpoint.openai.azure.com/",
-  "api_key": "your-api-key",
-  "component_type": "model"
+  "provider": "autogen_ext.models.openai.AzureOpenAIChatCompletionClient",
+  "component_type": "model",
+  "version": 1,
+  "component_version": 1,
+  "description": "Chat completion client for Azure OpenAI hosted models.",
+  "label": "AzureOpenAIChatCompletionClient",
+  "config": {
+    "model": "gpt-4o",
+    "api_key": "sk-...",
+    "azure_endpoint": "https://{your-custom-endpoint}.openai.azure.com/",
+    "azure_deployment": "{your-azure-deployment}",
+    "api_version": "2024-06-01"
+  }
 }
 ```
 
 Have a local model server like Ollama, vLLM or LMStudio that provide an OpenAI compliant endpoint? You can use that as well.
 
 ```json
 {
-  "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
-  "model_type": "OpenAIChatCompletionClient",
-  "base_url": "http://localhost:1234/v1",
-  "api_version": "1.0",
+  "provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
   "component_type": "model",
-  "model_capabilities": {
-    "vision": false,
-    "function_calling": true,
-    "json_output": false
+  "version": 1,
+  "component_version": 1,
+  "description": "Chat completion client for OpenAI hosted models.",
+  "label": "OpenAIChatCompletionClient",
+  "config": {
+    "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
+    "model_info": {
+      "vision": false,
+      "function_calling": true,
+      "json_output": false,
+      "family": "unknown"
+    },
+    "base_url": "http://localhost:1234/v1"
   }
 }
 ```
 
 ```{caution}
-It is important that you add the `model_capabilities` field to the model client specification for custom models. This is used by the framework instantiate and use the model correctly. Also, the `AssistantAgent` and many other agents in AgentChat require the model to have the `function_calling` capability.
+It is important that you add the `model_info` field to the model client specification for custom models. This is used by the framework instantiate and use the model correctly. Also, the `AssistantAgent` and many other agents in AgentChat require the model to have the `function_calling` capability.
 ```
 
 ## Q: The server starts but I can't access the UI

diff --git a/python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/index.md b/python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/index.md
@@ -12,7 +12,9 @@ myst:
 
 AutoGen Studio is a low-code interface built to help you rapidly prototype AI agents, enhance them with tools, compose them into teams and interact with them to accomplish tasks. It is built on [AutoGen AgentChat](https://microsoft.github.io/autogen) - a high-level API for building multi-agent applications.
 
-![AutoGen Studio](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)
+> See a video tutorial on AutoGen Studio v0.4 (02/25) - [https://youtu.be/oum6EI7wohM](https://youtu.be/oum6EI7wohM)
+
+[![A Friendly Introduction to AutoGen Studio v0.4](https://img.youtube.com/vi/oum6EI7wohM/maxresdefault.jpg)](https://www.youtube.com/watch?v=oum6EI7wohM)
 
 Code for AutoGen Studio is on GitHub at [microsoft/autogen](https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio)
 

diff --git a/python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/usage.md b/python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/usage.md
@@ -10,7 +10,9 @@ myst:
 AutoGen Studio provides a Team Builder interface where developers can define multiple components and behaviors. Users can create teams, add agents to teams, attach tools and models to agents, and define team termination conditions.
 After defining a team, users can test it in the Playground view to accomplish various tasks through direct interaction.
 
-![AutoGen Studio](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)
+> See a video tutorial on AutoGen Studio v0.4 (02/25) - [https://youtu.be/oum6EI7wohM](https://youtu.be/oum6EI7wohM)
+
+[![A Friendly Introduction to AutoGen Studio v0.4](https://img.youtube.com/vi/oum6EI7wohM/maxresdefault.jpg)](https://www.youtube.com/watch?v=oum6EI7wohM)
 
 ## Declarative Specification of Componenents
 
@@ -100,8 +102,6 @@ This example shows a team with a single agent, using the `RoundRobinGroupChat` t
 
 ## Building an Agent Team
 
-<div style="padding:58.13% 0 0 0;position:relative; border-radius:5px; border-bottom:10px"><iframe src="https://player.vimeo.com/video/1043133833?badge=0&amp;autopause=0&amp;player_id=0&amp;app_id=58479" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write; encrypted-media" style="position:absolute;top:0;left:0;width:100%;height:100%;" title="AutoGen Studio v0.4x - Drag and Drop Interface"></iframe></div><script src="https://player.vimeo.com/api/player.js"></script>
-
 <br/>
 
 AutoGen Studio integrates closely with all component abstractions provided by AutoGen AgentChat, including {py:class}`~autogen_agentchat.teams`, {py:class}`~autogen_agentchat.agents`, {py:class}`~autogen_core.models`, {py:class}`~autogen_core.tools`, and termination {py:class}`~autogen_agentchat.conditions`.
@@ -117,6 +117,8 @@ Team Builder Operations:
     - Agents: Add models and tools
 - Save team configurations
 
+Note: For each node in the visual builder, you can click on the edit icon (top right) to view and edit the JSON configuration.
+
 ## Gallery - Sharing and Reusing Components
 
 A Gallery is a collection of components - teams, agents, models, tools, and terminations - that can be shared and reused across projects.

diff --git a/python/packages/autogen-studio/autogenstudio/gallery/builder.py b/python/packages/autogen-studio/autogenstudio/gallery/builder.py
@@ -148,7 +148,7 @@ def create_default_gallery() -> Gallery:
     mistral_vllm_model = OpenAIChatCompletionClient(
         model="TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
         base_url="http://localhost:1234/v1",
-        model_info=ModelInfo(vision=False, function_calling=True, json_output=False),
+        model_info=ModelInfo(vision=False, function_calling=True, json_output=False, family="unknown"),
     )
     builder.add_model(
         mistral_vllm_model.dump_component(),
@@ -236,6 +236,7 @@ def create_default_gallery() -> Gallery:
         model_client=base_model,
         termination_condition=web_termination,
     )
+
     builder.add_team(
         websurfer_team.dump_component(),
         label="Web Agent Team (Operator)",
@@ -256,8 +257,8 @@ def create_default_gallery() -> Gallery:
 
     builder.add_tool(
         tools.fetch_webpage_tool.dump_component(),
-        label="Webpage Generation Tool",
-        description="A tool that generates a webpage from a list of images. Requires beautifulsoup4 html2text library to function.",
+        label="Fetch Webpage Tool",
+        description="A tool that fetches the content of a webpage and converts it to markdown. Requires the requests and beautifulsoup4 library to function.",
     )
 
     builder.add_tool(
@@ -272,6 +273,83 @@ def create_default_gallery() -> Gallery:
         description="A tool that performs Google searches using the Google Custom Search API. Requires the requests library, [GOOGLE_API_KEY, GOOGLE_CSE_ID] to be set,  env variable to function.",
     )
 
+    # Create deep research agent
+    model_client = OpenAIChatCompletionClient(model="gpt-4o", temperature=0.7)
+
+    research_assistant = AssistantAgent(
+        name="research_assistant",
+        description="A research assistant that performs web searches and analyzes information",
+        model_client=model_client,
+        tools=[tools.google_search_tool, tools.fetch_webpage_tool],
+        system_message="""You are a research assistant focused on finding accurate information.
+        Use the google_search tool to find relevant information.
+        Break down complex queries into specific search terms.
+        Always verify information across multiple sources when possible.
+        When you find relevant information, explain why it's relevant and how it connects to the query. When you get feedback from the a verifier agent, use your tools to act on the feedback and make progress.""",
+    )
+
+    verifier = AssistantAgent(
+        name="verifier",
+        description="A verification specialist who ensures research quality and completeness",
+        model_client=model_client,
+        system_message="""You are a research verification specialist.
+        Your role is to:
+        1. Verify that search queries are effective and suggest improvements if needed
+        2. Explore drill downs where needed e.g, if the answer is likely in a link in the returned search results, suggest clicking on the link
+        3. Suggest additional angles or perspectives to explore. Be judicious in suggesting new paths to avoid scope creep or wasting resources, if the task appears to be addressed and we can provide a report, do this and respond with "TERMINATE".
+        4. Track progress toward answering the original question
+        5. When the research is complete, provide a detailed summary in markdown format. For incomplete research, end your message with "CONTINUE RESEARCH". For complete research, end your message with APPROVED.
+        Your responses should be structured as:
+        - Progress Assessment
+        - Gaps/Issues (if any)
+        - Suggestions (if needed)
+        - Next Steps or Final Summary""",
+    )
+
+    summary_agent = AssistantAgent(
+        name="summary_agent",
+        description="A summary agent that provides a detailed markdown summary of the research as a report to the user.",
+        model_client=model_client,
+        system_message="""You are a summary agent. Your role is to provide a detailed markdown summary of the research as a report to the user. Your report should have a reasonable title that matches the research question and should summarize the key details in the results found in natural an actionable manner. The main results/answer should be in the first paragraph.
+        Your report should end with the word "TERMINATE" to signal the end of the conversation.""",
+    )
+
+    termination = TextMentionTermination("TERMINATE") | MaxMessageTermination(max_messages=30)
+
+    selector_prompt = """You are coordinating a research team by selecting the team member to speak/act next. The following team member roles are available:
+    {roles}.
+    The research_assistant performs searches and analyzes information.
+    The verifier evaluates progress and ensures completeness.
+    The summary_agent provides a detailed markdown summary of the research as a report to the user.
+
+    Given the current context, select the most appropriate next speaker.
+    The research_assistant should search and analyze.
+    The verifier should evaluate progress and guide the research (select this role is there is a need to verify/evaluate progress). You should ONLY select the summary_agent role if the research is complete and it is time to generate a report.
+
+    Base your selection on:
+    1. Current stage of research
+    2. Last speaker's findings or suggestions
+    3. Need for verification vs need for new information
+    Read the following conversation. Then select the next role from {participants} to play. Only return the role.
+
+    {history}
+
+    Read the above conversation. Then select the next role from {participants} to play. ONLY RETURN THE ROLE."""
+
+    deep_research_team = SelectorGroupChat(
+        participants=[research_assistant, verifier, summary_agent],
+        model_client=model_client,
+        termination_condition=termination,
+        selector_prompt=selector_prompt,
+        allow_repeated_speaker=True,
+    )
+
+    builder.add_team(
+        deep_research_team.dump_component(),
+        label="Deep Research Team",
+        description="A team that performs deep research using web searches, verification, and summarization.",
+    )
+
     return builder.build()
 
 

diff --git a/python/packages/autogen-studio/autogenstudio/version.py b/python/packages/autogen-studio/autogenstudio/version.py
@@ -1,3 +1,3 @@
-VERSION = "0.4.0"
+VERSION = "0.4.1"
 __version__ = VERSION
 APP_NAME = "autogenstudio"
diff --git a/python/packages/autogen-studio/frontend/src/components/sidebar.tsx b/python/packages/autogen-studio/frontend/src/components/sidebar.tsx
@@ -220,7 +220,7 @@ const Sidebar = ({ link, meta, isMobile }: SidebarProps) => {
                         ],
                       })
                     }
-                    className="group flex gap-x-3 rounded-md p-2 text-sm font-medium text-primary hover:text-accent hover:bg-secondary justify-center"
+                    className="group hidden flex gap-x-3 rounded-md p-2 text-sm font-medium text-primary hover:text-accent hover:bg-secondary justify-center"
                   >
                     <Settings className="h-6 w-6 shrink-0 text-secondary group-hover:text-accent" />
                   </Link>
-Original file line number
+Diff line change
@@ Expand Up @@
                             ],
                           })
                         }
-                        className="group flex gap-x-3 rounded-md p-2 text-sm font-medium text-primary hover:text-accent hover:bg-secondary justify-center"
+                        className="group hidden flex gap-x-3 rounded-md p-2 text-sm font-medium text-primary hover:text-accent hover:bg-secondary justify-center"
                       >
                         <Settings className="h-6 w-6 shrink-0 text-secondary group-hover:text-accent" />
                       </Link>
@@ Expand Down @@