Skip to main content

v1.63.11-stable

Krrish Dholakia
Krrish Dholakia
CEO, LiteLLM
Ishaan Jaffer
Ishaan Jaffer
CTO, LiteLLM

These are the changes since v1.63.2-stable.

This release is primarily focused on:

  • [Beta] Responses API Support
  • Snowflake Cortex Support, Amazon Nova Image Generation
  • UI - Credential Management, re-use credentials when adding new models
  • UI - Test Connection to LLM Provider before adding a model

Known Issues​

  • 🚨 Known issue on Azure OpenAI - We don't recommend upgrading if you use Azure OpenAI. This version failed our Azure OpenAI load test

Docker Run LiteLLM Proxy​

docker run
-e STORE_MODEL_IN_DB=True
-p 4000:4000
ghcr.io/berriai/litellm:main-v1.63.11-stable

Demo Instance​

Here's a Demo Instance to test changes:

New Models / Updated Models​

  • Image Generation support for Amazon Nova Canvas Getting Started
  • Add pricing for Jamba new models PR
  • Add pricing for Amazon EU models PR
  • Add Bedrock Deepseek R1 model pricing PR
  • Update Gemini pricing: Gemma 3, Flash 2 thinking update, LearnLM PR
  • Mark Cohere Embedding 3 models as Multimodal PR
  • Add Azure Data Zone pricing PR
    • LiteLLM Tracks cost for azure/eu and azure/us models

LLM Translation​

  1. New Endpoints
  1. New LLM Providers
  1. New LLM Features
  1. Bug Fixes
  • OpenAI: Return code, param and type on bad request error More information on litellm exceptions
  • Bedrock: Fix converse chunk parsing to only return empty dict on tool use PR
  • Bedrock: Support extra_headers PR
  • Azure: Fix Function Calling Bug & Update Default API Version to 2025-02-01-preview PR
  • Azure: Fix AI services URL PR
  • Vertex AI: Handle HTTP 201 status code in response PR
  • Perplexity: Fix incorrect streaming response PR
  • Triton: Fix streaming completions bug PR
  • Deepgram: Support bytes.IO when handling audio files for transcription PR
  • Ollama: Fix "system" role has become unacceptable PR
  • All Providers (Streaming): Fix String data: stripped from entire content in streamed responses PR

Spend Tracking Improvements​

  1. Support Bedrock converse cache token tracking Getting Started
  2. Cost Tracking for Responses API Getting Started
  3. Fix Azure Whisper cost tracking Getting Started

UI​

Re-Use Credentials on UI​

You can now onboard LLM provider credentials on LiteLLM UI. Once these credentials are added you can re-use them when adding new models Getting Started

Test Connections before adding models​

Before adding a model you can test the connection to the LLM provider to verify you have setup your API Base + API Key correctly

General UI Improvements​

  1. Add Models Page
    • Allow adding Cerebras, Sambanova, Perplexity, Fireworks, Openrouter, TogetherAI Models, Text-Completion OpenAI on Admin UI
    • Allow adding EU OpenAI models
    • Fix: Instantly show edit + deletes to models
  2. Keys Page
    • Fix: Instantly show newly created keys on Admin UI (don't require refresh)
    • Fix: Allow clicking into Top Keys when showing users Top API Key
    • Fix: Allow Filter Keys by Team Alias, Key Alias and Org
    • UI Improvements: Show 100 Keys Per Page, Use full height, increase width of key alias
  3. Users Page
    • Fix: Show correct count of internal user keys on Users Page
    • Fix: Metadata not updating in Team UI
  4. Logs Page
    • UI Improvements: Keep expanded log in focus on LiteLLM UI
    • UI Improvements: Minor improvements to logs page
    • Fix: Allow internal user to query their own logs
    • Allow switching off storing Error Logs in DB Getting Started
  5. Sign In/Sign Out

Security​

  1. Support for Rotating Master Keys Getting Started
  2. Fix: Internal User Viewer Permissions, don't allow internal_user_viewer role to see Test Key Page or Create Key Button More information on role based access controls
  3. Emit audit logs on All user + model Create/Update/Delete endpoints Getting Started
  4. JWT
    • Support multiple JWT OIDC providers Getting Started
    • Fix JWT access with Groups not working when team is assigned All Proxy Models access
  5. Using K/V pairs in 1 AWS Secret Getting Started

Logging Integrations​

  1. Prometheus: Track Azure LLM API latency metric Getting Started
  2. Athina: Added tags, user_feedback and model_options to additional_keys which can be sent to Athina Getting Started

Performance / Reliability improvements​

  1. Redis + litellm router - Fix Redis cluster mode for litellm router PR

General Improvements​

  1. OpenWebUI Integration - display thinking tokens
  • Guide on getting started with LiteLLM x OpenWebUI. Getting Started
  • Display thinking tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) Getting Started

Complete Git Diff​

Here's the complete git diff