Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.61.1
What's Changed
- Show Guardrails on UI by @krrishdholakia in #8447
- Log applied guardrails on LLM API call by @krrishdholakia in #8452
- Ui Fixes Teams Setting #8347 by @tahaali-dev in #8353
- (UI) allow adding model aliases for teams by @ishaan-jaff in #8471
- (round 4 fixes) - Team model alias setting by @ishaan-jaff in #8474
Full Changelog: v1.61.0...v1.61.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 160.0 | 180.272351294557 | 6.268555221678184 | 0.0 | 1874 | 0 | 118.979319999994 | 3618.562145999988 |
| Aggregated | Passed ✅ | 160.0 | 180.272351294557 | 6.268555221678184 | 0.0 | 1874 | 0 | 118.979319999994 | 3618.562145999988 |
Contributors
krrishdholakia, ishaan-jaff, and tahaali-dev
Assets 4
v1.61.0.dev1
What's Changed
- (Feat) - Allow calling Nova models on
/bedrock/invoke/by @ishaan-jaff in #8397 - Run litellm in dev mode by @CakeCrusher in #8404
- (Bug Fix) - Bedrock completions with aws_region_name by @ishaan-jaff in #8384
- added gemini 2.0 models to docs by @mubashir1osmani in #8412
- Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192) by @tahaali-dev in #8357
- Revert "Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192)" by @krrishdholakia in #8416
- Allow editing model api key + provider on UI by @krrishdholakia in #8406
- Allow org admin to create teams on UI by @krrishdholakia in #8407
New Contributors
- @CakeCrusher made their first contribution in #8404
Full Changelog: v1.60.8...v1.61.0.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.0.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 239.66008324484403 | 6.157355937828466 | 0.0033427556665735435 | 1842 | 1 | 171.46882700001242 | 4024.462443999994 |
| Aggregated | Passed ✅ | 210.0 | 239.66008324484403 | 6.157355937828466 | 0.0033427556665735435 | 1842 | 1 | 171.46882700001242 | 4024.462443999994 |
Contributors
krrishdholakia, ishaan-jaff, and 3 other contributors
Assets 4
v1.61.0
What's Changed
- (Feat) - Allow calling Nova models on
/bedrock/invoke/by @ishaan-jaff in #8397 - Run litellm in dev mode by @CakeCrusher in #8404
- (Bug Fix) - Bedrock completions with aws_region_name by @ishaan-jaff in #8384
- added gemini 2.0 models to docs by @mubashir1osmani in #8412
- Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192) by @tahaali-dev in #8357
- Revert "Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192)" by @krrishdholakia in #8416
- Allow editing model api key + provider on UI by @krrishdholakia in #8406
- Allow org admin to create teams on UI by @krrishdholakia in #8407
- Azure api version check - fix str compare to convert to int by @krrishdholakia in #8438
- Fix callback add when user_config passed + support passing openai org client-side by @krrishdholakia in #8443
- Org UI Improvements by @krrishdholakia in #8436
- (e2e testing) - add tests for using litellm
/team/updates in multi-instance deployments with Redis by @ishaan-jaff in #8440 - (Feat) - Allow viewing Request/Response Logs stored in GCS Bucket by @ishaan-jaff in #8449
New Contributors
- @CakeCrusher made their first contribution in #8404
Full Changelog: v1.60.8...v1.61.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 180.0 | 213.86169773089247 | 6.297834462789351 | 0.003342799608699231 | 1884 | 1 | 81.07622899996159 | 4173.802059999957 |
| Aggregated | Passed ✅ | 180.0 | 213.86169773089247 | 6.297834462789351 | 0.003342799608699231 | 1884 | 1 | 81.07622899996159 | 4173.802059999957 |
Contributors
krrishdholakia, ishaan-jaff, and 3 other contributors
Assets 4
v1.60.8
What's Changed
- UI Updates by @krrishdholakia in #8345
- OIDC Scope based model access by @krrishdholakia in #8343
- Fix azure max retries error by @krrishdholakia in #8340
- Update deepseek API prices for 2025-02-08 by @Winston-503 in #8363
- fix(nvidia_nim/embed.py): add 'dimensions' support by @krrishdholakia in #8302
- fix: dictionary changed size during iteration error (#8327) by @krrishdholakia in #8341
- fix: add azure/o1-2024-12-17 to model_prices_and_context_window.json by @byrongrogan in #8371
- (Security fix) Mask redis pwd on
/cache/ping+ add timeout value and elapsed time on azure + http calls by @krrishdholakia in #8377 - Handle azure deepseek reasoning response (#8288) by @krrishdholakia in #8366
- Anthropic Citations API Support by @krrishdholakia in #8382
- (Feat) - Add
/bedrock/invokesupport for all Anthropic models by @ishaan-jaff in #8383 - O3 mini native streaming support by @krrishdholakia in #8387
Full Changelog: v1.60.6...v1.60.8
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.8
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 170.0 | 189.56173781509457 | 6.206468643400922 | 0.0 | 1855 | 0 | 149.30551800000558 | 3488.08786699999 |
| Aggregated | Passed ✅ | 170.0 | 189.56173781509457 | 6.206468643400922 | 0.0 | 1855 | 0 | 149.30551800000558 | 3488.08786699999 |
Contributors
krrishdholakia, ishaan-jaff, and 2 other contributors
Assets 4
v1.60.4-stable
Full Changelog: v1.60.2-dev1...v1.60.4-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.4-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 140.0 | 150.928871417902 | 6.3094717994281515 | 0.0 | 1888 | 0 | 115.36740400003964 | 821.5439119999814 |
| Aggregated | Passed ✅ | 140.0 | 150.928871417902 | 6.3094717994281515 | 0.0 | 1888 | 0 | 115.36740400003964 | 821.5439119999814 |
v1.60.6
What's Changed
- Azure OpenAI improvements - o3 native streaming, improved tool call + response format handling by @krrishdholakia in #8292
- Fix edit team on ui by @krrishdholakia in #8295
- Improve rpm check on keys by @krrishdholakia in #8301
- docs: fix enterprise links by @wagnerjt in #8294
- Add gemini-2.0-flash pricing + model info by @krrishdholakia in #8303
- Add Arize Cookbook for Turning on LiteLLM Proxy by @exiao in #8336
- Add aistudio GEMINI 2.0 to model_prices_and_context_window.json by @dceluis in #8335
- Fix pricing for Gemini 2.0 Flash 001 by @elabbarw in #8320
- [DOCS] Update local_debugging.md by @rokbenko in #8308
- (Bug Fix - Langfuse) - fix for when model response has
choices=[]by @ishaan-jaff in #8339 - Fixed meta llama 3.3 key for Databricks API by @anton164 in #8093
- fix(utils.py): handle key error in msg validation by @krrishdholakia in #8325
- (bug fix router.py) - safely handle
choices=[]on llm responses by @ishaan-jaff in #8342 - (QA+UI) - e2e flow for adding assembly ai passthrough endpoints by @ishaan-jaff in #8337
New Contributors
- @exiao made their first contribution in #8336
- @dceluis made their first contribution in #8335
- @rokbenko made their first contribution in #8308
- @anton164 made their first contribution in #8093
Full Changelog: v1.60.5...v1.60.6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 217.05167674521235 | 6.288425886864887 | 0.0 | 1880 | 0 | 164.17646499996863 | 2306.284880000021 |
| Aggregated | Passed ✅ | 200.0 | 217.05167674521235 | 6.288425886864887 | 0.0 | 1880 | 0 | 164.17646499996863 | 2306.284880000021 |
Contributors
anton164, exiao, and 6 other contributors
Assets 4
v1.60.5-dev1
What's Changed
- Azure OpenAI improvements - o3 native streaming, improved tool call + response format handling by @krrishdholakia in #8292
- Fix edit team on ui by @krrishdholakia in #8295
- Improve rpm check on keys by @krrishdholakia in #8301
- docs: fix enterprise links by @wagnerjt in #8294
- Add gemini-2.0-flash pricing + model info by @krrishdholakia in #8303
Full Changelog: v1.60.5...v1.60.5-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.5-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 244.25501266865584 | 6.165773096861878 | 0.006687389475989022 | 1844 | 2 | 149.75326000001132 | 28799.14171600001 |
| Aggregated | Passed ✅ | 190.0 | 244.25501266865584 | 6.165773096861878 | 0.006687389475989022 | 1844 | 2 | 149.75326000001132 | 28799.14171600001 |
Contributors
wagnerjt and krrishdholakia
Assets 4
v1.60.5
What's Changed
- Added a guide for users who want to use LiteLLM with AI/ML API. by @waterstark in #7058
- Added compatibility guidance, etc. for xAI Grok model by @zhaohan-dong in #8282
- (Security fix) - remove code block that inserts master key hash into DB by @ishaan-jaff in #8268
- (UI) - Add Assembly AI provider to UI by @ishaan-jaff in #8297
- (feat) - Add Assembly AI to model cost map by @ishaan-jaff in #8298
- fixed issues #8126 and #8127 (#8275) by @ishaan-jaff in #8299
- (Refactor) - migrate bedrock invoke to
BaseLLMHTTPHandlerclass by @ishaan-jaff in #8290
New Contributors
- @waterstark made their first contribution in #7058
Full Changelog: v1.60.4...v1.60.5
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.5
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 251.44053604962153 | 6.19421782055854 | 0.0 | 1854 | 0 | 167.35073600000305 | 4496.06190000003 |
| Aggregated | Passed ✅ | 210.0 | 251.44053604962153 | 6.19421782055854 | 0.0 | 1854 | 0 | 167.35073600000305 | 4496.06190000003 |
Contributors
ishaan-jaff, zhaohan-dong, and waterstark
Assets 4
v1.60.4
What's Changed
- Internal User Endpoint - vulnerability fix + response type fix by @krrishdholakia in #8228
- Litellm UI fixes 8123 v2 (#8208) by @ishaan-jaff in #8245
- Update model_prices_and_context_window.json by @superpoussin22 in #8249
- Update model_prices_and_context_window.json by @superpoussin22 in #8256
- (dependency) - pip loosen httpx version requirement by @ishaan-jaff in #8255
- Add hyperbolic deepseek v3 model configurations by @lowjiansheng in #8232
- fix(prometheus.py): fix setting key budget metrics by @krrishdholakia in #8234
- (feat) - add supports tool choice to model cost map by @ishaan-jaff in #8265
- (feat) - track org_id in SpendLogs by @ishaan-jaff in #8253
- (Bug fix) - Langfuse / Callback settings stored in DB by @ishaan-jaff in #8251
- Fix passing top_k parameter for Bedrock Anthropic models (#8131) by @ishaan-jaff in #8269
- (Feat) - Add support for structured output on
bedrock/novamodels + add utillitellm.supports_tool_choiceby @ishaan-jaff in #8264 - [BETA] Support OIDC
rolebased access to proxy by @krrishdholakia in #8260 - Fix deepseek calling - refactor to use base_llm_http_handler by @krrishdholakia in #8266
- allows dynamic message redaction by @krrishdholakia in #8270
Full Changelog: v1.60.2...v1.60.4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 243.98647747354212 | 6.187158959524932 | 0.0033407985742575225 | 1852 | 1 | 94.81396500007122 | 3976.009301999966 |
| Aggregated | Passed ✅ | 210.0 | 243.98647747354212 | 6.187158959524932 | 0.0033407985742575225 | 1852 | 1 | 94.81396500007122 | 3976.009301999966 |
Contributors
lowjiansheng, krrishdholakia, and 2 other contributors
Assets 4
v1.60.2-dev1
What's Changed
- Internal User Endpoint - vulnerability fix + response type fix by @krrishdholakia in #8228
- Litellm UI fixes 8123 v2 (#8208) by @ishaan-jaff in #8245
- Update model_prices_and_context_window.json by @superpoussin22 in #8249
- Update model_prices_and_context_window.json by @superpoussin22 in #8256
- (dependency) - pip loosen httpx version requirement by @ishaan-jaff in #8255
- Add hyperbolic deepseek v3 model configurations by @lowjiansheng in #8232
- fix(prometheus.py): fix setting key budget metrics by @krrishdholakia in #8234
- (feat) - add supports tool choice to model cost map by @ishaan-jaff in #8265
- (feat) - track org_id in SpendLogs by @ishaan-jaff in #8253
- (Bug fix) - Langfuse / Callback settings stored in DB by @ishaan-jaff in #8251
- Fix passing top_k parameter for Bedrock Anthropic models (#8131) by @ishaan-jaff in #8269
- (Feat) - Add support for structured output on
bedrock/novamodels + add utillitellm.supports_tool_choiceby @ishaan-jaff in #8264 - [BETA] Support OIDC
rolebased access to proxy by @krrishdholakia in #8260 - Fix deepseek calling - refactor to use base_llm_http_handler by @krrishdholakia in #8266
- allows dynamic message redaction by @krrishdholakia in #8270
Full Changelog: v1.60.2...v1.60.2-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.2-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 220.0 | 251.01715871561 | 6.124144736848293 | 0.0 | 1832 | 0 | 171.2837300000274 | 3691.155395999999 |
| Aggregated | Passed ✅ | 220.0 | 251.01715871561 | 6.124144736848293 | 0.0 | 1832 | 0 | 171.2837300000274 | 3691.155395999999 |
Contributors
lowjiansheng, krrishdholakia, and 2 other contributors