v1.61.20.rc

What's Changed

• Add new gpt-4.5-preview model + other updates by @krrishdholakia in #8879
• (fix) Anthropic pass through cost tracking by @ishaan-jaff in #8874
• (Bug Fix) - Accurate token counting for /anthropic/ API Routes on LiteLLM Proxy by @ishaan-jaff in #8880
• (fix) Pass through spend tracking - ensure custom_llm_provider is tracked for Vertex, Google AI Studio, Anthropic by @ishaan-jaff in #8882

Full Changelog: v1.61.13-stable...v1.61.20.rc

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.20.rc

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	271.87043339059113	6.110384289826154	0.003342660990058071	1828	1	218.3162040000184	3401.295629999993
Aggregated	Passed ✅	250.0	271.87043339059113	6.110384289826154	0.003342660990058071	1828	1	218.3162040000184	3401.295629999993

v1.61.20-nightly

What's Changed

• vertex ai anthropic thinking param support + cost calculation for vertex_ai/claude-3-7-sonnet by @krrishdholakia in #8853

Full Changelog: v1.61.19-nightly...v1.61.20-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.20-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	274.35423830943404	6.203695089639108	0.0033443100213687914	1855	1	99.50650100000757	3744.788088000007
Aggregated	Passed ✅	250.0	274.35423830943404	6.203695089639108	0.0033443100213687914	1855	1	99.50650100000757	3744.788088000007

v1.61.19.dev1

What's Changed

• vertex ai anthropic thinking param support + cost calculation for vertex_ai/claude-3-7-sonnet by @krrishdholakia in #8853

Full Changelog: v1.61.19-nightly...v1.61.19.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.19.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	279.81049794793904	6.161760602979549	0.0	1844	0	217.23908899997468	4265.23701100001
Aggregated	Passed ✅	250.0	279.81049794793904	6.161760602979549	0.0	1844	0	217.23908899997468	4265.23701100001

v1.61.19-nightly

What's Changed

• fix(get_litellm_params.py): handle no-log being passed in via kwargs by @krrishdholakia in #8830
• fix(o_series_transformation.py): fix optional param check for o-serie… by @krrishdholakia in #8787
• chore: set ttlSecondsAfterFinished on the migration job in the litellm-helm chart by @ashwin153 in #8593
• Litellm dev bedrock anthropic 3 7 v2 by @krrishdholakia in #8843
• Mark Claude Haiku 3.5 as vision-capable by @minhduc0711 in #8840
• feat: enhance migrations job with additional configurable properties by @mknet3 in #8636
• (UI + Backend) Fix Adding Azure, Azure AI Studio models on LiteLLM by @ishaan-jaff in #8856
• fix caching on main branch by @krrishdholakia in #8858
• [Bug]: Deepseek error on proxy after upgrading to 1.61.13-stable by @ishaan-jaff in #8860

New Contributors

• @ashwin153 made their first contribution in #8593
• @mknet3 made their first contribution in #8636

Full Changelog: v1.61.17-nightly...v1.61.19-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.19-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	190.0	197.9691385267413	6.185555293319119	0.003341737057438746	1851	1	53.4934159999807	967.1408859999815
Aggregated	Passed ✅	190.0	197.9691385267413	6.185555293319119	0.003341737057438746	1851	1	53.4934159999807	967.1408859999815

v1.61.17.dev1

What's Changed

• fix(get_litellm_params.py): handle no-log being passed in via kwargs by @krrishdholakia in #8830
• fix(o_series_transformation.py): fix optional param check for o-serie… by @krrishdholakia in #8787
• chore: set ttlSecondsAfterFinished on the migration job in the litellm-helm chart by @ashwin153 in #8593

New Contributors

• @ashwin153 made their first contribution in #8593

Full Changelog: v1.61.17-nightly...v1.61.17.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	170.0	197.91492646891462	6.342832402753434	6.342832402753434	1898	1898	138.25161099998695	4268.093897000028
Aggregated	Failed ❌	170.0	197.91492646891462	6.342832402753434	6.342832402753434	1898	1898	138.25161099998695	4268.093897000028

v1.61.17-nightly

What's Changed

• (UI) Fixes for managing Internal Users by @ishaan-jaff in #8786
• Litellm contributor prs 02 24 2025 by @krrishdholakia in #8781
• (UI) Allow adding MSFT SSO on UI by @ishaan-jaff in #8779
• (UI) Minor fix, clear new team form after adding a new team by @ishaan-jaff in #8776
• [ui] Icons on navbar profile dropdown by @Mte90 in #8792
• fix(UI): model name overflow in model hub cards by @Aditya-A-G in #8749
• fix vertex_ai claude 3.7 naming by @emerzon in #8807
• (Router) - If allowed_fails or allowed_fail_policy set, use that for single deployment cooldown logic by @ishaan-jaff in #8668
• (Bug fix) - reading /parsing request body when on hypercorn by @ishaan-jaff in #8734
• (Bug fix) - running litellm proxy on wndows by @ishaan-jaff in #8735
• Anthropic model cost map updates by @krrishdholakia in #8816
• Adding Azure Phi-4 by @emerzon in #8808
• (Bug Fix) Using LiteLLM Python SDK with model=litellm_proxy/ for embedding, image_generation, transcription, speech, rerank by @ishaan-jaff in #8815
• (Bug fix) - allow using Assistants GET, DELETE on /openai pass through routes by @ishaan-jaff in #8818
• (Bug fix) dd-trace used by default on litellm proxy by @ishaan-jaff in #8817
• Adding openrouter claude-3.7-sonnet by @fengjiajie in #8826
• (UI) - Create Key flow for existing users by @ishaan-jaff in #8844

New Contributors

• @Mte90 made their first contribution in #8792
• @Aditya-A-G made their first contribution in #8749

Full Changelog: v1.61.16-nightly...v1.61.17-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	110.0	132.37492176765105	6.343856763236461	6.343856763236461	1898	1898	93.89094700003398	3315.9179240000185
Aggregated	Failed ❌	110.0	132.37492176765105	6.343856763236461	6.343856763236461	1898	1898	93.89094700003398	3315.9179240000185

v1.61.16-nightly

What's Changed

• fix: remove aws params from bedrock embedding request body (#8618) by @ishaan-jaff in #8696
• Add anthropic3-7-sonnet by @dragosMC91 in #8766
• fix incorrect variable name in reliability section of docs by @niinpatel in #8753
• Litellm contributor prs 02 24 2025 by @krrishdholakia in #8775
• Add anthropic thinking + reasoning content support by @krrishdholakia in #8778

New Contributors

• @niinpatel made their first contribution in #8753

Full Changelog: v1.61.15-nightly...v1.61.16-nightly

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.16-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	150.0	173.09079325414655	6.248259928705226	6.248259928705226	1869	1869	131.67032300003711	1529.8640780000028
Aggregated	Failed ❌	150.0	173.09079325414655	6.248259928705226	6.248259928705226	1869	1869	131.67032300003711	1529.8640780000028

v1.61.15-nightly

What's Changed

• Add cost tracking for rerank via bedrock + jina ai by @krrishdholakia in #8691
• add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support by @krrishdholakia in #8684
• Correct spelling in user_management_heirarchy.md by @oaustegard in #8716
• (Feat) - UI, Allow sorting models by Created_At and all other columns on the UI by @ishaan-jaff in #8725
• (UI) Edit Model flow improvements by @ishaan-jaff in #8729
• Support arize phoenix on litellm proxy (#7756) by @krrishdholakia in #8715
• fix(amazon_deepseek_transformation.py): remove from stream o… by @krrishdholakia in #8717
• Add cohere v2/rerank support (#8421) by @krrishdholakia in #8605
• fix(proxy/_types.py): fixes issue where internal user able to escalat… by @krrishdholakia in #8740

New Contributors

• @oaustegard made their first contribution in #8716

Full Changelog: v1.61.13-nightly...v1.61.15-nightly

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.15-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	140.0	149.21232461729608	6.452882828983183	6.452882828983183	1931	1931	114.22628599996187	662.278525000005
Aggregated	Failed ❌	140.0	149.21232461729608	6.452882828983183	6.452882828983183	1931	1931	114.22628599996187	662.278525000005

v1.61.13-stable

Full Changelog: v1.61.13-nightly...v1.61.13-stable

🚨 Known Issue:

• DD Trace was enabled by default on litellm docker: #8788
• Expect a patched v1.61.13-stable with the fix

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.61.13-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	180.0	204.35323738803527	6.313934995711798	6.313934995711798	1889	1889	146.12962300003574	2180.2391240000247
Aggregated	Failed ❌	180.0	204.35323738803527	6.313934995711798	6.313934995711798	1889	1889	146.12962300003574	2180.2391240000247

v1.55.8-stable-patched

Full Changelog: v1.55.8-stable...v1.55.8-stable-patched

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm_stable_release_branch-v1.55.8-stable-patched

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	150.0	172.39167931961987	6.334948735889217	6.334948735889217	1896	1896	131.01931900001773	2316.6445349999663
Aggregated	Failed ❌	150.0	172.39167931961987	6.334948735889217	6.334948735889217	1896	1896	131.01931900001773	2316.6445349999663