Python: Improve/Simplify the Assistant Agents

[moonbox3]

Motivation and Context

The current abstractions we have around the Semantic Kernel Assistant Agents are considered to be "fully-wrapped." This is beneficial in that SK provides a tight/complete abstraction over the agent; however, it doesn't scale well when new features are added to the underlying APIs (OpenAI) - it requires updates from SK to make each and every update. It also adds a lot more complexity in terms of needing to handle everything in SK, requires more code coverage, and chances for bugs to surface due to these complexities.

NOTE: this update is breaking. We're moving to simplify the abstractions/APIs before we move Assistant Agents from experimental to GA.

Similar to how the AzureAIAgent was created, we're moving to a "less-wrapped" abstraction around the OpenAI Assistant v2 APIs. We do have some "convenience methods" on the Agents, via class methods, to configure things like the code interpreter tool and resource, file search tool and resource, the clients required to interact with the Assistants, as well as being able to easily configure structured outputs, similar to how it is done with a ChatCompletion (either provide the Pydantic model or the Python class).

To work with Assistant Agents now, it's as simple as:

from semantic_kernel.agents.open_ai import AzureAssistantAgent
from semantic_kernel.functions import kernel_function

class MenuPlugin:
   # Plugin code here using the @kernel_function decorator

# Create the client using Azure OpenAI resources and configuration
client, model = AzureAssistantAgent.setup_resources()

# Create the assistant definition
definition = await client.beta.assistants.create(
    model=model,
    instructions="Answer questions about the menu.",
    name="Host",
)

agent = AzureAssistantAgent(
    client=client,
    definition=definition,
    plugins=[MenuPlugin()],  # The plugins can be passed in as a list to the constructor
)

# Note: plugins can also be configured on the Kernel and passed in as a parameter to the OpenAIAssistantAgent

# Define a thread and invoke the agent with the user input
thread = await agent.client.beta.threads.create()

I will create a migration guide (issue) showing what needs to be updated from the previous usage to this new pattern.

Description

This PR provides the new abstractions for the AzureAssistantAgent/OpenAIAssistantAgent classes.

• Updates underlying code
• Updates samples showing new patterns
• Updates unit tests
• Updates READMEs.
• Closes Python: Simplify the AzureAssistantAgent/OpenAIAssistantAgent #10605

Samples will be updated accordingly to show new patterns.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[Copilot]

Copilot reviewed 58 out of 58 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

python/samples/concepts/agents/assistant_agent/assistant_agent_streaming.py:54

• The 'MenuPlugin' is referenced but not defined or imported; please ensure it is properly defined or imported in this file.

plugins=[MenuPlugin()]

[eavanvalkenburg]

do we need to check if a plugin already exists?

[moonbox3]

I want to document that if providing plugins via the constructor, they will take precedence over any existing plugins (with the same name) that are contained in the kernel (if passed in).