Litellm Rust

litellm-rust

[](https://github.com/avivsinai/litellm-rust/actions/workflows/ci.yml) [](https://crates.io/crates/litellm-rust) [](https://docs.rs/litellm-rust) [](LICENSE) [](https://www.rust-lang.org/)

Minimal Rust SDK port of [LiteLLM](https://github.com/BerriAI/litellm) (library only). Provides a unified interface for chat, embeddings, images, and video across multiple LLM providers.

**Note**: This project is under active development. APIs may change.

Features

• Unified client for OpenAI-compatible, Anthropic, Gemini, xAI, and Z.AI providers
• Chat completions with streaming (SSE) support
• Tool calling — model-requested function calls surfaced on ChatResponse.tool_calls in OpenAI shape (Anthropic tool_use and Gemini functionCall normalized through the same shape)
• Text embeddings
• Image generation (OpenAI DALL-E / GPT Image)
• Video generation (Gemini Veo, OpenAI Sora)
• Model routing with provider/model format
• Automatic retry with exponential backoff
• Cost tracking via response headers
• Embedded model registry with pricing and context window data
• Daily model registry refreshes from LiteLLM's canonical catalog

Supported Providers

Provider	Chat	Streaming	Embeddings	Images	Video
OpenAI-compatible	yes	yes	yes	yes	yes
Anthropic	yes	yes	-	-	-
Gemini	yes	-	-	yes	yes
xAI	yes	yes	-	-	-
Z.AI	yes	yes	-	-	-

Installation

Prefer [crates.io](https://crates.io/crates/litellm-rust). The published crate is **litellm-rust** `0.3.0`.