Bedrock models don't work because of inference profiles are required

[puffitos]

Description

Problem

Catwalk's bedrock.json lists models using their foundation model IDs (e.g.
anthropic.claude-sonnet-4-6). These IDs are what aws bedrock list-foundation-models returns. However, the Bedrock runtime API does not
accept foundation model IDs directly — it requires inference profile IDs
(e.g. eu.anthropic.claude-sonnet-4-6), which are what aws bedrock list-inference-profiles returns.

Any consumer using catwalk's bedrock model IDs verbatim should get the following
error from the Bedrock API, at least in the eu-central region (see following section):

bad request: POST "https://bedrock-runtime.eu-central-1.amazonaws.com/v1/messages":
400 Bad Request {"message":"The provided model identifier is invalid."}

Observed behavior (eu-central-1)

The following is based on direct experience using crush with an AWS account
hosted in eu-central-1. Other regions may behave differently and have not
been verified, but the underlying cause is the same for all non-global regions.

Initially the problem was intermittent. Normal prompting appeared to work, but
secondary model calls would silently fail with the 400 error above:

• Session title generation would fail and fall back silently:

  error generating title with small model; trying big model
  

• Agentic fetch tool calls would fail entirely:

  agentic_fetch: error generating response —
  bad request: 400 Bad Request {"message":"The provided model identifier is invalid."}
  

After a recent crush update, the problem became consistent — even primary
prompts started returning 400 errors, making the tool unusable with the
bedrock provider. At that point the root cause was investigated and traced back
to the model IDs catwalk provides.

The full error chain observed in the logs:

{"level":"ERROR","msg":"error generating title with small model; trying big model",
 "err":"bad request: POST \"https://bedrock-runtime.eu-central-1.amazonaws.com/v1/messages\":
 400 Bad Request {\"message\":\"The provided model identifier is invalid.\"}"}

{"level":"ERROR","msg":"agentic_fetch: error generating response",
 "error":"bad request: POST \"https://bedrock-runtime.eu-central-1.amazonaws.com/v1/messages\":
 400 Bad Request {\"message\":\"The provided model identifier is invalid.\"}"}

And on startup after the catwalk update that removed the region prefix logic:

Failed to configure selected models: failed to select default models:
default large model anthropic.claude-sonnet-4-6 not found for provider bedrock

Background

AWS Bedrock requires cross-region inference profile IDs for all API calls.
These IDs carry a region-group prefix that determines which AWS infrastructure
the request is routed through:

Region	Prefix	Example
us-*, ca-central-1	us.	us.anthropic.claude-sonnet-4-6
eu-*	eu.	eu.anthropic.claude-sonnet-4-6
ap-northeast-1	jp.	jp.anthropic.claude-sonnet-4-6
ap-southeast-2	au.	au.anthropic.claude-sonnet-4-6
All regions	global.	global.anthropic.claude-sonnet-4-6

Some regions (e.g. ap-northeast-2, sa-east-1) only have global. profiles
and no regional prefix.

Catwalk's model list mirrors list-foundation-models, which returns bare IDs
with no prefix. These IDs are not directly callable anywhere — every region
requires a prefixed inference profile ID.

Proposed Solutions

Option 1 — Consumer-side prefix (current workaround)

The consumer reads AWS_REGION, derives the prefix, and prepends it to all
model IDs and default model references. Catwalk exposes helper methods to make
this safe and atomic:

// PrefixModelIDs prepends prefix to all model IDs and default model references.
func (p *Provider) PrefixModelIDs(prefix string)

// ApplyBedrockRegion derives and applies the correct prefix for a given AWS region.
func (p *Provider) ApplyBedrockRegion(region string)

Usage in the consumer:

p.ApplyBedrockRegion(os.Getenv("AWS_REGION"))
prepared.Models = p.Models

Advantages:

• Less invasive code change.
• Simpler to maintain.

Drawbacks:

• Requires AWS_REGION to be set; does not handle regions that only have
  global. profiles; the prefix mapping logic must be kept in sync with AWS
  as new regions are added.
• Adds new exported functions which require consumers to update their code.

Option 2 — Catwalk serves inference profile IDs directly (preferred)

Add a regions field to each model in bedrock.json listing the available
inference profile prefixes (e.g. ["us", "eu", "global"]). Catwalk reads
AWS_REGION at startup and resolves each model's ID to the correct inference
profile, falling back to the global profile when no regional one is available.

{
  "id": "bedrock",
  "default_large_model_id": "global.anthropic.claude-sonnet-4-6",
  "default_small_model_id": "global.anthropic.claude-haiku-4-5-20251001-v1:0",
  "models": [
    {
      "id": "anthropic.claude-sonnet-4-6",
      "regions": ["us", "eu", "global"],
      ...
    }
  ]
}

The consumer receives a Provider with model IDs already resolved to the
correct inference profile for the user's region. No transformation is needed
on the consumer side. If AWS_REGION is not set, global.* profiles are
returned, which work in all regions.

Advantages:

• The model list is self-contained and correct out of the box.
• Consumers do not need to know anything about inference profile prefixes.
• No consumer code changes required.

Drawbacks:

• The regions list per model requires manual maintenance as AWS adds new
  regional profiles

I've implemented both, but will open an MR only for the second, as it seems like the better one. I wonder why am I the only one suffering from this bug; maybe it's because most people just go to anthropic directly for their claude usage and don't use AWS for that :)

Version

v0.30.3

Environment

WSL/Windows Terminal

[puffitos]

Update: Inference profiles are mandatory for all current Anthropic models

Tested direct model ID invocation vs inference profile prefixed IDs using a simple Python script against us-east-1 with a personal AWS account (no custom inference profiles configured).

Results

Model	Direct (no prefix)	With us. prefix
anthropic.claude-sonnet-4-6	❌ ValidationException	✅ Works
anthropic.claude-haiku-4-5-20251001-v1:0	❌ ValidationException	✅ Works
anthropic.claude-opus-4-20250514-v1:0	❌ ValidationException	✅ Works*
anthropic.claude-3-5-sonnet-20241022-v2:0	❌ ValidationException	❌ Legacy model
anthropic.claude-3-haiku-20240307-v1:0	❌ Legacy model	❌ Legacy model

Every current-gen model returns this error when invoked without a prefix:

ValidationException: Invocation of model ID <id> with on-demand throughput isn't supported. Retry your request with the ID or ARN of an inference profile that contains this model.

Key takeaways

1. Inference profiles are required, not optional. Direct model ID invocation is rejected for all current Anthropic models on Bedrock.
2. System-defined inference profiles work out of the box — no account setup needed beyond IAM permissions. The us., eu., global. prefixed IDs are pre-created by AWS.
3. Not all models have a global profile. Older Opus models (4, 4.1) only have us. — they genuinely cannot be used outside US regions via cross-region inference - we should fallback to us for those.
4. This was tested from us-east-1 (not the usual working region), confirming the behavior isn't region-specific.

This further validates the approach of always resolving model IDs to inference profile IDs with region prefixes.

* works after providing a use case to anthropic