How Yelp Built a Back-Testing Engine for Safer, Smarter Ad Budget Allocation Samuele Mazzanti, Applied Scientist Feb 2, 2026 Introduction Modern advertising platforms are fast-paced and interconnected: even small adjustments...

Yelp's platform is a central hub for consumers and businesses, offering insights into local businesses, user reviews, and recommendations. Through articles, guides, and user-generated content, Yelp offers insights into discovering restaurants, shops, and services in various locations. Readers can learn about reading and writing reviews, finding popular businesses, and exploring local communities to make informed decisions about where to eat, shop, or visit.

Yelp Engineering

Yelp developed a back-testing engine to safely simulate changes to their ad budget allocation system before deploying them to production. The system replays historical campaign data day-by-day using production code, ML models (CatBoost) to predict outcomes like clicks and leads, and Bayesian optimization (Scikit-Optimize) to explore parameter spaces. This approach allows faster iteration, reduces A/B testing risks, catches bugs early, and provides more accurate impact predictions than traditional methods. The engine integrates eight components including Git submodules for production code, Redshift for historical data, and MLFlow for experiment tracking.

How Yelp Built a Back-Testing Engine for Safer, Smarter Ad Budget Allocation

How our Ad Budget Allocation system works