computer use for anthropic on bedrock #1177
Conversation
|
DescriptionSummary By MatterAI
🔄 What ChangedThis Pull Request introduces enhanced support for various tool types, including 'computer' use, for Anthropic models when accessed via Bedrock. The 🔍 Impact of the ChangeThis change significantly improves the flexibility and compatibility of the gateway with Anthropic's advanced tool capabilities, such as 'computer-use'. It allows users to define and pass various tool types dynamically, making the system more adaptable to future tool extensions without requiring code changes. The added null check in Bedrock's tool processing enhances robustness, and the refined return logic for tool configurations ensures that tool-related parameters are only included when tools are actually present. This directly addresses the issue of supporting 📁 Total Files Changed
Total files changed: 4 🧪 Test AddedManual testing was performed using a sample request body: {
"model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic_beta":"computer-use-2025-01-24",
"max_tokens": 2000,
"tools": ,
"messages":
}This test verifies the end-to-end flow of passing 🔒Security VulnerabilitiesNo new security vulnerabilities were detected in this pull request. The changes primarily focus on feature enhancement and robustness. MotivationThis PR addresses the issue of supporting 'computer use' and other generic tool types for Anthropic models when accessed via Bedrock, as detailed in #1176. Type of Change
How Has This Been Tested?
Screenshots (if applicable)N/A Checklist
Related IssuesTip Quality Recommendations
Sequence DiagramsequenceDiagram
participant User
participant GatewayAPI as Gateway API Endpoint
participant AnthropicChatComplete as Anthropic Chat Complete Config
participant BedrockChatComplete as Bedrock Chat Complete Config
participant BedrockUtils as Bedrock Utils
participant AnthropicAPI as Anthropic API
participant BedrockAPI as Bedrock API
User->>GatewayAPI: POST /chat/completions (with tools array)
GatewayAPI->>BedrockChatComplete: Process Bedrock Request (params)
BedrockChatComplete->>BedrockChatComplete: Iterate params.tools
alt If tool.function exists
BedrockChatComplete->>BedrockChatComplete: Add { toolSpec: { name, description, inputSchema } } to tools array
end
alt If tool.cache_control exists and not Amazon model
BedrockChatComplete->>BedrockChatComplete: Add { cachePoint: { type: 'ephemeral' } } to tools array
end
BedrockChatComplete-->>GatewayAPI: Return toolConfig (if tools.length > 0) or null
GatewayAPI->>BedrockUtils: Call transformAnthropicAdditionalModelRequestFields(params)
BedrockUtils->>BedrockUtils: Iterate params.tools (Tool[])
alt If tool.type is not 'function'
BedrockUtils->>BedrockUtils: Get toolOptions = tool[tool.type]
BedrockUtils->>BedrockUtils: Construct Anthropic-specific tool object
BedrockUtils->>BedrockUtils: Add { ...toolOptions, name: tool.type, type: toolOptions?.name, cache_control } to anthropicTools array
end
alt If anthropicTools array is not empty
BedrockUtils->>BedrockUtils: Add anthropicTools to additionalModelRequestFields['tools']
end
BedrockUtils-->>GatewayAPI: Return additionalModelRequestFields
GatewayAPI->>AnthropicChatComplete: Process Anthropic Request (params)
AnthropicChatComplete->>AnthropicChatComplete: Iterate params.tools
alt If tool.function exists
AnthropicChatComplete->>AnthropicChatComplete: Add { name, description, input_schema, cache_control } to tools array
end
alt If tool.type exists (e.g., 'computer')
AnthropicChatComplete->>AnthropicChatComplete: Get toolOptions = tool[tool.type]
AnthropicChatComplete->>AnthropicChatComplete: Add { ...toolOptions, name: tool.type, type: toolOptions?.name, cache_control } to tools array
end
AnthropicChatComplete->>AnthropicAPI: Send Chat Completion Request (with transformed tools)
AnthropicAPI-->>AnthropicChatComplete: Response
AnthropicChatComplete-->>GatewayAPI: Response
GatewayAPI->>BedrockAPI: Send Converse API Request (with transformed tools/additional fields)
BedrockAPI-->>GatewayAPI: Response
GatewayAPI-->>User: Return API Response
|
![matter-code-review[bot]](https://avatars.githubusercontent.com/in/474549?s=60&v=4){kind=small}
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
#1176
Testing:
sample request body
{ "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "anthropic_beta":"computer-use-2025-01-24", "max_tokens": 2000, "tools": [ { "type": "computer", "computer": { "name": "computer_20250124", "display_width_px": 1024, "display_height_px": 768, "display_number": 1 } } // { // "type": "bash", // "bash": { // "name": "bash_20250124" // } // } // } ], "messages": [ { "role": "user", "content": "make a curl request to google" } ] }