I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing

A developer shares how replacing GPT-4 with a local Qwen2.5-7B model via Ollama eliminated 23 CI/CD pipeline failures caused by non-deterministic LLM outputs. The core insight: temperature=0 reduces variance in hosted APIs but doesn't guarantee it, while a locally-run model with a fixed seed produces identical outputs every run. The post covers the full journey from GPT-4 failures (inconsistent JSON keys, markdown fences, type mismatches) through failed mitigations (prompt engineering, cleanup parsers, function calling) to the final Ollama-based solution with GitHub Actions integration and Pydantic validation. Tradeoffs are acknowledged: GPT-4 handles ambiguous and multilingual documents better, but for structured extraction on well-formed documents, a 7B model is sufficient and far more reliable.