I’ve been using LiteLLM for about 8 months now. It worked fine for the first few weeks, but once I moved past prototyping, I started hitting problems that I couldn’t ignore anymore.
<h2>The Issues I Ran Into</h2>
Import time is painfully slow. Every time my serverless function cold starts, it takes 3+ seconds just to import the library. Not to make an API call - just to load the thing. Turns out the <code>__init__.py</code> file has 1,200+ lines of imports loading every provider SDK. I only use OpenAI and Anthropic but I’m loading Google, AWS, Azure, everything.
Performance tanks under load. I thought it was my code at first. Then I realized at around 400-500 requests per second, the gateway just starts falling apart. P99 latency goes from a few seconds to 90+ seconds. I’ve seen other people mention this on GitHub issues too.
Memory usage keeps climbing. Even after they fixed the memory leaks, I’m still seeing 350-400MB of RAM usage. For a proxy that’s supposed to sit between my app and the LLM providers, that seems excessive.
The codebase is rough. I had to debug something last month and found that <code>main.py</code> is over 5,500 lines. One file. Finding what I needed took forever. The whole thing uses global state for configuration, which makes it impossible to use different settings for different parts of my app.
Breaking changes in minor updates. I pinned a version, tested everything, deployed. Two weeks later I tried to upgrade and things broke. No warning in the release notes. Just… broke.
<h2>We switched to Bifrost - <a href="https://github.com/maximhq/bifrost" target="_blank" rel="noopener nofollow">https://github.com/maximhq/bifrost</a></h2>
Someone mentioned Bifrost in a Discord server. I saw their <a href="https://docs.getbifrost.ai/benchmarking/getting-started" target="_blank" rel="noopener nofollow">benchmarks</a> comparing it to LiteLLM - 54x faster P99, way lower memory usage. Seemed too good to be true, so I tested it myself.
Cold starts went to zero. It’s a compiled Go binary instead of Python. My serverless functions went from 3-4 second delays to instant. Under load at 1,000 RPS, it didn’t break. P99 stayed under 2 seconds. Memory stayed at 120MB.
Migration was straightforward - changed my base URL, kept my existing code. Been running it in production for 6 weeks now. Performance is consistent.
Bifrost is newer with a smaller community, so that’s a tradeoff. But for my setup, it just works better.
If you’re hitting similar issues, test it yourself with your actual traffic. Don’t take anyone’s word for it.

AI Engineering

LiteLLM faces production issues including 3+ second cold starts from loading all provider SDKs, performance degradation at 400-500 RPS with P99 latency reaching 90+ seconds, 350-400MB memory usage, a 5,500-line main.py file with global state, and breaking changes in minor updates. Bifrost, a Go-based alternative, reportedly delivers 54x faster P99 latency, zero cold starts, consistent performance at 1,000 RPS, and 120MB memory footprint.

LiteLLM Has Some Serious Issues in Production