Skip to content

Conversation

@roomote
Copy link

@roomote roomote bot commented Oct 21, 2025

Description

This PR attempts to address Issue #8750 by adding support for AWS Bedrock's Global Inference feature, which automatically routes requests to the optimal AWS region for supported models.

Changes

  • ✅ Added awsUseGlobalInference setting to provider configuration schema
  • ✅ Implemented global. prefix support in Bedrock provider for supported models
  • ✅ Added UI checkbox for global inference in Bedrock settings (similar to cross-region inference)
  • ✅ Prioritized global inference over cross-region when both are enabled
  • ✅ Added comprehensive test suite with 21 test cases
  • ✅ Support for global inference in custom ARN parsing

Supported Models

The following models support global inference:

  • Claude Sonnet 4 (anthropic.claude-sonnet-4-20250514-v1:0)
  • Claude Sonnet 4.5 (anthropic.claude-sonnet-4-5-20250929-v1:0)
  • Claude Opus 4 (anthropic.claude-opus-4-20250514-v1:0)
  • Claude Opus 4.1 (anthropic.claude-opus-4-1-20250805-v1:0)
  • Claude 3.7 Sonnet (anthropic.claude-3-7-sonnet-20250219-v1:0)
  • Claude Haiku 4.5 (anthropic.claude-haiku-4-5-20251001-v1:0)

Testing

  • ✅ All new tests passing (21 test cases)
  • ✅ No regressions in existing Bedrock tests
  • ✅ Type checking passes
  • ✅ Linting passes

UI Experience

The new global inference checkbox:

  • Only appears for models that support global inference
  • Disables cross-region inference when enabled (global takes priority)
  • Follows the same UI pattern as the existing cross-region inference option

Fixes #8750

Feedback and guidance are welcome!


Important

Adds global inference support for AWS Bedrock models, prioritizing it over cross-region inference, with UI updates and comprehensive tests.

  • Behavior:
    • Adds awsUseGlobalInference setting to provider-settings.ts for AWS Bedrock models.
    • Implements global. prefix support in AwsBedrockHandler for supported models.
    • Prioritizes global inference over cross-region inference when both are enabled.
    • Supports global inference in custom ARN parsing in AwsBedrockHandler.
  • UI:
    • Adds checkbox for global inference in Bedrock.tsx, disabling cross-region inference when enabled.
  • Testing:
    • Adds bedrock-global-inference.spec.ts with 21 test cases for global inference behavior.
  • Models:
    • Updates bedrock.ts to include models supporting global inference.
  • Misc:
    • Updates settings.json for UI text related to global inference.

This description was created by Ellipsis for 9b4c326. You can customize this summary. It will automatically update as commits are pushed.

- Add awsUseGlobalInference setting to provider settings schema
- Add support for global. prefix in Bedrock provider for supported models
- Add UI checkbox for global inference in Bedrock settings
- Prioritize global inference over cross-region when both are enabled
- Add comprehensive tests for global inference functionality
- Support global inference in custom ARN parsing

Supported models:
- Claude Sonnet 4, 4.5
- Claude Opus 4, 4.1
- Claude 3.7 Sonnet
- Claude Haiku 4.5

Fixes #8750
@roomote roomote bot requested review from cte, jr and mrubens as code owners October 21, 2025 17:16
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 21, 2025
@roomote
Copy link
Author

roomote bot commented Oct 21, 2025

Reviewing your PR now—feedback coming shortly! 🚀

Follow Along on Roo Code Cloud

@dosubot dosubot bot added the enhancement New feature or request label Oct 21, 2025
Comment on lines +451 to +458
export const BEDROCK_GLOBAL_INFERENCE_MODEL_IDS = [
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-haiku-4-5-20251001-v1:0",
] as const
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the issue comment from @ronyblum, only Sonnet 4, Sonnet 4.5, and Haiku 4.5 support Global Inference. However, this constant includes Opus 4, Opus 4.1, and Claude 3.7 Sonnet which do not support global inference. Including unsupported models will cause the code to incorrectly apply the global. prefix to models that don't support it, potentially causing API errors. The list should only include: anthropic.claude-sonnet-4-20250514-v1:0, anthropic.claude-sonnet-4-5-20250929-v1:0, and anthropic.claude-haiku-4-5-20251001-v1:0.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Oct 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

[ENHANCEMENT] Global Inference for Bedrock models supported

2 participants