Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] - BerriAI/litellm

Dickson A.

Community Picks is a section on daily.dev where our community members share the most interesting and valuable content they've discovered online. From insightful articles to handy tools, every post is a gem curated by our dedicated coomunity. To contribute to Community Picks, you need to have at least 250 reputation points, ensuring that only active and trusted members can share their finds.

Community Picks

BerriAI's LiteLLM is a Python SDK and proxy server that allows users to call over 100 LLM APIs using the OpenAI format. It provides features like translation of inputs, consistent output, retry/fallback logic, and budget/rate limits per project. It supports synchronous, asynchronous, and streaming responses. The proxy server enables load balancing and tracking of expenditures across projects. The latest stable release can be installed via Docker, and the system supports various logging and observability tools.

BerriAI/litellm: Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format

LiteLLM Proxy Server (LLM Gateway) - ( Docs )