From Print to Digital: Making Weekly Flyers Shoppable at Instacart Through Computer Vision and LLMs Key contributors: Prithvi Srinivasan, Shishir Kumar Prasad, Kristen Morgan, Bryan Pham, Rick …

Instacart's resource offers insights, tutorials, and resources for shoppers and grocery delivery drivers. Readers can learn about grocery shopping tips, delivery strategies, and customer service best practices. With articles, guides, and tutorials, Instacart helps shoppers make the most out of their shopping experience and earn income through delivery services.

Instacart

Instacart built a two-phase ML pipeline to automate digitization of grocery flyers, reducing processing time from 3-4 hours to 30 minutes. Phase 1 uses Meta's Segment Anything Model with custom algorithms for image segmentation, achieving 75-90% accuracy in extracting product bounding boxes. Phase 2 combines OCR, LLMs, and Instacart's existing search infrastructure to match segmented images to catalog products with 95% recall. The system handles complex flyer layouts through techniques like Weighted Boxes Fusion, model ensembling, and ML-based filtering.

From Print to Digital: Making Weekly Flyers Shoppable at Instacart Through Computer Vision and LLMs

Get Prithvi Srinivasan ’s stories in your inbox