Releases: PurpleAILAB/Decepticon
Releases · PurpleAILAB/Decepticon
Release list
v1.1.38
v1.1.37
v1.1.36
v1.1.34
v1.1.33
v1.1.29 — litellm auth/* prompt caching
Wires up Anthropic prompt caching on the Claude Code OAuth (auth/*) LiteLLM path — both the cache-token metering and the actual conversation-history caching. Long, tool-heavy runs now re-read the repeated context prefix at 0.1× instead of full price.
What changed
- Meter cache tokens. The OAuth handler (
config/claude_code_handler.py) now surfacescache_creation_input_tokens/cache_read_input_tokensfrom the Anthropic response in the returned usage (non-streaming and streaming), and includes them inprompt_tokensso LiteLLM's cost path prices cache reads (0.1×) / writes (1.25×) instead of silently dropping them. - Cache the conversation, not just the system prompt. A
cache_controlbreakpoint is now stamped on the last message block (incremental / auto-advancing), so the whole system + tool-result / history prefix is cached and re-read across turns. System breakpoints capped to 3 to stay within Anthropic's global limit of 4.
Impact
Previously only the system prompt carried cache_control and the response's cache counters were discarded, so caching was invisible/unpriced and the dominant tool-result/history bulk was never cached. This release closes both gaps.
Verification (live, real Anthropic via OAuth)
- System caching: call 1
cache_creation=11427→ call 2cache_read=11427. - Multi-turn history caching: Turn A
cache_creation=9086→ Turn Bcache_read=9086, cache_creation=35(full prefix served at 0.1×, only the new delta written).
Downstream
Consumers that build FROM ghcr.io/purpleailab/decepticon-litellm:1.1.29 must also add cache pricing (cache_read_input_token_cost / cache_creation_input_token_cost) to any auth/* model_info that overrides the built-in cost map, or cache tokens price at $0.
Commits
- feat(litellm): meter + cache the auth/* OAuth path (prompt caching) (#748)