A practical guide to running LLMs locally using Ollama to avoid sending sensitive data to cloud providers. Covers installation, pulling models, calling Ollama from Python via its native SDK, the OpenAI-compatible SDK, and LangChain integration. Demonstrates a provider-agnostic factory pattern for switching between cloud and local models, and shows how to build a LangGraph ReAct agent backed by a local model. Uses FinanceGPT (a personal finance app processing bank statements and tax forms) as a real-world example. Also discusses tradeoffs including response quality, inference speed, hardware requirements, and function calling reliability.
Table of contents
Table of ContentsPrerequisitesWhat is Ollama?How Ollama's API worksHow to Call Ollama from PythonHow to Integrate Ollama into a LangChain AppHow to Build an LLM-Provider Agnostic AppHow to use Ollama with LangGraphHow FinanceGPT Uses This in PracticeTradeoffs to be Aware OfConclusionCheck Out FinanceGPTResourcesSort: