Ollama is still the easiest way to start local LLMs, but it's the worst way to keep running them

Ollama is widely recommended as the easiest entry point for running LLMs locally, but it comes with significant drawbacks for long-term use. It produces fewer tokens per second than running llama.cpp directly, ships with a very low default context window (2048 tokens) that isn't obvious to beginners, and uses a proprietary Show more