Building production AI agents on Elasticsearch: 5 key lessons

After a year and over one million messages across five AI tools, Elastic's Field Technology team shares five production lessons from their agentic RAG system built on Elasticsearch. Key findings: interaction logs are a strategic asset for measuring AI quality; adoption follows a power law with ~8% of users generating 80% of sessions; partial context retrieval (tangentially relevant documents) produces worse answers (8.15/10) than returning no context at all (9.18/10); zero retrieval results should be treated as knowledge gap signals rather than failures; and high token counts in deep sessions correlate with higher quality scores (9.74/10), not user struggle. The core takeaway is that retrieval relevance and confidence thresholds matter more than LLM selection.