Skip to content

fix(bedrock): cache system prompt in auto mode - #3681

Draft
arielnabavian wants to merge 1 commit into
strands-agents:mainfrom
arielnabavian:feat/cache-01-bedrock-system
Draft

fix(bedrock): cache system prompt in auto mode#3681
arielnabavian wants to merge 1 commit into
strands-agents:mainfrom
arielnabavian:feat/cache-01-bedrock-system

Conversation

@arielnabavian

Copy link
Copy Markdown
Contributor

Summary

CacheConfig(strategy="auto") previously only wrote a cache breakpoint on the last user message, leaving the (usually largest and most static) system prefix uncached. For workloads that share a system prompt across calls with varying user messages, every call wrote to cache and none read — a net cost regression versus no caching.

Appends a cachePoint to the system prompt when caching resolves to "anthropic" for the model and the caller hasn't already placed one at the end. Both SDKs mirror the change. TS adds a systemTTL knob to BedrockCacheConfig to match toolsTTL/messagesTTL.

Fixes #3144. First of a 9-PR stack that closes #2970.

Stack

This is 1/9 in a stack that adds caching to every Strands model provider that can support it, culminating in default-on caching for BedrockModel (PR 10).

  1. fix(bedrock): cache system prompt in auto modethis PR
  2. feat(litellm): route cache_config per underlying
  3. feat(openai): cache_config + prompt_cache_key
  4. feat(llamaapi): cache_config + prompt_cache_key/retention
  5. feat(mistral): cache_config + prompt_cache_key
  6. feat(openai-responses): prompt_cache_options for GPT-5.6
  7. feat(vercel): accept cacheConfig, warn on per-block-marker providers
  8. feat(gemini): cache_config no-op acceptance
  9. feat(bedrock): default cache_config to auto

Coordinated with — not duplicating — open PR #3571 (AnthropicModel cache_config + cache_tools).

Test plan

  • All 189 Python bedrock tests pass
  • All 178 TS bedrock tests pass
  • mypy, ruff, prettier, eslint clean
  • Real-API smoke test on Claude Sonnet 4.5 + Opus 4.6 + Haiku 4.5 (owner to run before merge)

CacheConfig(strategy="auto") previously only wrote a cache breakpoint on
the last user message, leaving the (usually largest and most static)
system prefix uncached. For workloads that share a system prompt across
calls with varying user messages, every call wrote to cache and none
read — a net cost regression versus no caching.

Append a cachePoint to the system prompt when caching resolves to
"anthropic" for the model and the caller hasn't already placed one at
the end. Both SDKs mirror the change. TS adds a systemTTL knob to
BedrockCacheConfig to match toolsTTL/messagesTTL.

Fixes strands-agents#3144.
@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added the size/m label Aug 6, 2026
@yonib05 yonib05 added complexity/high A touched function exceeds cognitive complexity 25; may be worth splitting size/s and removed size/m labels Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

complexity/high A touched function exceeds cognitive complexity 25; may be worth splitting size/s

Projects

None yet

2 participants