v1.59.10

What's Changed

• (UI) - View Logs Page - Refinement by @ishaan-jaff in #8087
• (Feat) pass through vertex - allow using credentials defined on litellm router for vertex pass through by @ishaan-jaff in #8100
• (UI) Allow using a model / credentials for pass through routes by @ishaan-jaff in #8099
• ui - fix chat ui tab sending model param by @ishaan-jaff in #8105
• Litellm dev 01 29 2025 p1 by @krrishdholakia in #8097
• Support new bedrock/converse_like/<model> route by @krrishdholakia in #8102
• feat(databricks/chat/transformation.py): add tools and 'tool_choice' param support by @krrishdholakia in #8076

Full Changelog: v1.59.9...v1.59.10

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.10

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	210.0	239.24647793068146	6.21745665443628	0.00334092243655899	1861	1	73.25327600000264	3903.3159660000083
Aggregated	Passed ✅	210.0	239.24647793068146	6.21745665443628	0.00334092243655899	1861	1	73.25327600000264	3903.3159660000083

v1.59.8-stable

Full Changelog: v1.57.8-stable...v1.59.8-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.59.8-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	260.0	291.2207591958183	6.075260080470321	0.0	1818	0	223.10552599998346	3813.1267819999266
Aggregated	Passed ✅	260.0	291.2207591958183	6.075260080470321	0.0	1818	0	223.10552599998346	3813.1267819999266

v1.59.9

What's Changed

• Fix custom pricing - separate provider info from model info by @krrishdholakia in #7990
• Litellm dev 01 25 2025 p4 by @krrishdholakia in #8006
• (UI) - Adding new models enhancement - show provider logo by @ishaan-jaff in #8033
• (UI enhancement) - allow onboarding wildcard models on UI by @ishaan-jaff in #8034
• add openrouter/deepseek/deepseek-r1 by @paul-gauthier in #8038
• (UI) - allow assigning wildcard models to a team / key by @ishaan-jaff in #8041
• Add smolagents by @aymeric-roucher in #8026
• (UI) fixes to add model flow by @ishaan-jaff in #8043
• github - run stale issue/pr bot by @ishaan-jaff in #8045
• (doc) Add nvidia as provider by @raspawar in #8023
• feat(handle_jwt.py): initial commit adding custom RBAC support on jwt… by @krrishdholakia in #8037
• fix(utils.py): handle failed hf tokenizer request during calls by @krrishdholakia in #8032
• Bedrock document processing fixes by @krrishdholakia in #8005
• Fix bedrock model pricing + add unit test using bedrock pricing api by @krrishdholakia in #7978
• Add openai metadata param preview support + new x-litellm-timeout request header by @krrishdholakia in #8047
• (beta ui - spend logs view fixes & Improvements 1) by @ishaan-jaff in #8062
• (fix) - proxy reliability, ensure duplicate callbacks are not added to proxy by @ishaan-jaff in #8067
• (UI) Fixes for Adding model page - keep existing page as default, have 2nd tab for wildcard models by @ishaan-jaff in #8073

New Contributors

• @aymeric-roucher made their first contribution in #8026
• @raspawar made their first contribution in #8023

Full Changelog: v1.59.8...v1.59.9

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.9

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	270.0	301.01550717582927	6.14169679840119	0.0	1837	0	234.85362500002793	3027.238808999982
Aggregated	Failed ❌	270.0	301.01550717582927	6.14169679840119	0.0	1837	0	234.85362500002793	3027.238808999982

v1.59.8-dev1

What's Changed

• Fix custom pricing - separate provider info from model info by @krrishdholakia in #7990
• Litellm dev 01 25 2025 p4 by @krrishdholakia in #8006
• (UI) - Adding new models enhancement - show provider logo by @ishaan-jaff in #8033
• (UI enhancement) - allow onboarding wildcard models on UI by @ishaan-jaff in #8034
• add openrouter/deepseek/deepseek-r1 by @paul-gauthier in #8038
• (UI) - allow assigning wildcard models to a team / key by @ishaan-jaff in #8041
• Add smolagents by @aymeric-roucher in #8026
• (UI) fixes to add model flow by @ishaan-jaff in #8043
• github - run stale issue/pr bot by @ishaan-jaff in #8045
• (doc) Add nvidia as provider by @raspawar in #8023
• feat(handle_jwt.py): initial commit adding custom RBAC support on jwt… by @krrishdholakia in #8037
• fix(utils.py): handle failed hf tokenizer request during calls by @krrishdholakia in #8032

New Contributors

• @aymeric-roucher made their first contribution in #8026
• @raspawar made their first contribution in #8023

Full Changelog: v1.59.8...v1.59.8-dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.8-dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	230.0	253.74562668371757	6.073890684010945	0.0	1818	0	198.74819999995452	1957.5085989999934
Aggregated	Passed ✅	230.0	253.74562668371757	6.073890684010945	0.0	1818	0	198.74819999995452	1957.5085989999934

v1.59.8

What's Changed

• refactor: cleanup dead codeblock by @krrishdholakia in #7936
• add type annotation for litellm.api_base (#7980) by @krrishdholakia in #7994
• (QA / testing) - Add unit testing for key model access checks by @ishaan-jaff in #7999
• (Prometheus) - emit key budget metrics on startup by @ishaan-jaff in #8002
• (Feat) set guardrails per team by @ishaan-jaff in #7993
• Supported nested json schema on anthropic calls via proxy + fix langfuse sync sdk issues by @krrishdholakia in #8003
• Bug fix - [Bug]: If you create a key tied to a user that does not belong to a team, and then edit the key to add it to a team (the user is still not a part of a team), using that key results in an unexpected error by @ishaan-jaff in #8008
• (QA / testing) - Add e2e tests for key model access auth checks by @ishaan-jaff in #8000
• (Fix) langfuse - setting LANGFUSE_FLUSH_INTERVAL by @ishaan-jaff in #8007

Full Changelog: v1.59.7...v1.59.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	280.0	325.48398318207154	6.003526201462839	0.0	1796	0	234.56590200004257	3690.442290999954
Aggregated	Failed ❌	280.0	325.48398318207154	6.003526201462839	0.0	1796	0	234.56590200004257	3690.442290999954

v1.59.7

What's Changed

• Add datadog health check support + fix bedrock converse cost tracking w/ region name specified by @krrishdholakia in #7958
• Retry for replicate completion response of status=processing (#7901) by @krrishdholakia in #7965
• Ollama ssl verify = False + Spend Logs reliability fixes by @krrishdholakia in #7931
• (Feat) - allow setting default_on guardrails by @ishaan-jaff in #7973
• (Testing) e2e testing for team budget enforcement checks by @ishaan-jaff in #7988
• (UI) - Usage page show days when spend is 0 and round spend figures on charts to 2 sig figs by @ishaan-jaff in #7991
• (Feat) - Add GCS Pub/Sub Logging integration for sending DB SpendLogs to BigQuery by @ishaan-jaff in #7976
• fix(spend_tracking_utils.py): revert api key pass through fix by @krrishdholakia in #7977
• Ensure base_model cost tracking works across all endpoints by @krrishdholakia in #7989
• (UI) Allow admin to expose teams for joining by @krrishdholakia in #7992

Full Changelog: v1.59.6...v1.59.7

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.7

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	260.0	294.5630730660492	6.1254059494010225	0.0	1832	0	231.04980300001898	2728.9633709999634
Aggregated	Passed ✅	260.0	294.5630730660492	6.1254059494010225	0.0	1832	0	231.04980300001898	2728.9633709999634

v1.59.6

What's Changed

• Add attempted-retries and timeout values to response headers + more testing by @krrishdholakia in #7926
• Refactor prometheus e2e test by @yujonglee in #7919
• (Testing + Refactor) - Unit testing for team and virtual key budget checks by @ishaan-jaff in #7945
• docs: fix typo by @wagnerjt in #7953
• (Feat) - Allow Admin UI users to view spend logs even when not storing messages / responses by @ishaan-jaff in #7952
• (UI) - Set/edit guardrails on a virtual key by @ishaan-jaff in #7954
• (Feat) - emit litellm_team_budget_reset_at_metric and litellm_api_key_budget_remaining_hours_metric on prometheus by @ishaan-jaff in #7946
• (Feat) allow setting guardrails on a team on the API by @ishaan-jaff in #7959
• (UI) Set guardrails on Team Create and Edit page by @ishaan-jaff in #7963
• (GCS fix) - don't truncate payload by @ishaan-jaff in #7964
• Litellm dev 01 23 2025 p2 by @krrishdholakia in #7962

New Contributors

• @wagnerjt made their first contribution in #7953

Full Changelog: v1.59.5...v1.59.6

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.6

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	250.0	302.94444351157557	6.065526445072595	0.0	1814	0	184.99327999995785	3192.1896389999915
Aggregated	Failed ❌	250.0	302.94444351157557	6.065526445072595	0.0	1814	0	184.99327999995785	3192.1896389999915

v1.59.5

What's Changed

• Deepseek r1 support + watsonx qa improvements by @krrishdholakia in #7907
• (Testing) - Add e2e testing for langfuse logging with tags by @ishaan-jaff in #7922
• build(deps): bump undici from 6.21.0 to 6.21.1 in /docs/my-website by @dependabot in #7902
• (test) add e2e test for proxy with fallbacks + custom fallback message by @krrishdholakia in #7933
• (feat) - add deepseek/deepseek-reasoner to model cost map by @ishaan-jaff in #7935
• fix(utils.py): move adding custom logger callback to success event in… by @krrishdholakia in #7905
• Add provider_specifc_header param by @krrishdholakia in #7932
• (Refactor) Langfuse - remove prepare_metadata, langfuse python SDK now handles non-json serializable objects by @ishaan-jaff in #7925

Full Changelog: v1.59.3...v1.59.5

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.5

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	210.0	227.08635060543418	6.150672112760015	0.0	1840	0	180.76872099999264	2652.4827009999967
Aggregated	Passed ✅	210.0	227.08635060543418	6.150672112760015	0.0	1840	0	180.76872099999264	2652.4827009999967

v1.59.3.dev1

What's Changed

• Deepseek r1 support + watsonx qa improvements by @krrishdholakia in #7907
• (Testing) - Add e2e testing for langfuse logging with tags by @ishaan-jaff in #7922
• build(deps): bump undici from 6.21.0 to 6.21.1 in /docs/my-website by @dependabot in #7902
• (test) add e2e test for proxy with fallbacks + custom fallback message by @krrishdholakia in #7933

Full Changelog: v1.59.3...v1.59.3.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.3.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	230.0	259.2853146928995	6.073999238925992	0.0	1817	0	211.11294400003544	2538.129180999988
Aggregated	Passed ✅	230.0	259.2853146928995	6.073999238925992	0.0	1817	0	211.11294400003544	2538.129180999988

v1.59.3

What's Changed

• Update MLflow calllback and documentation by @B-Step62 in #7809

Full Changelog: v1.59.2...v1.59.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	229.9985951234699	6.27846665942667	0.0	1879	0	179.09318400000984	3769.753647000016
Aggregated	Passed ✅	200.0	229.9985951234699	6.27846665942667	0.0	1879	0	179.09318400000984	3769.753647000016