Run LLMs Locally with Docker: A Quickstart Guide to Model Runner

Running large language models (LLMs) locally can be challenging. Docker Model Runner, now available in Beta with Docker Desktop 4.40 for macOS on Apple silicon, simplifies this process by enabling easy pulling, running, and experimentation with LLMs on local machines. It features GPU acceleration, integration with the OpenAI API, and a collection of popular models available as standard OCI artifacts. The guide provides steps to enable Model Runner, use its CLI, and integrate it into applications.