A practical guide to building a structured LLM integration layer in Rails applications. Covers designing a BaseLLMRequest class with subclasses for each LLM call, using ERB templates for prompts, a Runner class for instrumentation (response time, token tracking, error handling), and two-level testing strategy: fast unit tests with mocked HTTP and scheduled prompt tests against real models. Also discusses three context strategies: rich context via prompt variables, tool use via function calling, and RAG for large data sets.
Table of contents
Why we need a layerLibrary choiceA quick tour of ruby_llmDesigning the base classWriting testsProviding LLMs access to our dataSort: