Building The Intent Engine: How Instacart is Revamping Query Understanding with LLMs Yuanzheng Zhu, Guanghua Shu, Raochuan Fan, Vinesh Gudla, Tejaswi Tenneti Introduction When people search for items …

Instacart's resource offers insights, tutorials, and resources for shoppers and grocery delivery drivers. Readers can learn about grocery shopping tips, delivery strategies, and customer service best practices. With articles, guides, and tutorials, Instacart helps shoppers make the most out of their shopping experience and earn income through delivery services.

Instacart

Instacart rebuilt their query understanding system using LLMs to better handle long-tail searches and ambiguous queries. The team progressed from context-engineering with RAG to fine-tuning smaller models like Llama-3-8B, consolidating multiple specialized models into a unified system. They implemented a hybrid architecture: an offline pipeline generates high-quality training data and caches results for common queries, while a fine-tuned real-time model handles rare searches. Through adapter merging, GPU optimization, and quantization experiments, they reduced latency from 700ms to 300ms while improving search quality metrics by 6% for tail queries.

Building The Intent Engine: How Instacart is Revamping Query Understanding with LLMs

Challenges in Traditional Query Understanding

Get Yuanzheng Zhu’s stories in your inbox

Building a New Foundation: Fine-Tuning for Real-Time Inference

The Path to Production: Taming Real-Time Latency