Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces Written by Amber Wang and Yoonji Kim at Lyft. Background Whenever you use the Lyft app, there is a …

LyftEng's platform is a central hub for engineering insights and technology updates from Lyft's engineering team. Through articles, tech talks, and open-source contributions, LyftEng offers insights into engineering challenges, innovation projects, and best practices in software development. Readers can learn about Lyft's engineering culture, technology stack, and contributions to the broader engineering community.

Lyft Engineering

Lyft's Foundational Models team describes a methodology for estimating long-term, market-mediated effects of pricing and incentive decisions in a two-sided marketplace. Standard A/B tests can't capture these effects because policy changes ripple through supply and demand in complex ways. The framework uses a two-step surrogacy approach: first, residualized regression models map policy changes to shifts in negative user experiences (wait times, surge, driver idleness); second, doubly-robust AIPW causal estimation maps those experience shifts to future behavioral outcomes via a surrogacy index. Both steps are validated with switch-back and user-split experiments respectively. Finally, direct and market-mediated long-term effects are combined and verified end-to-end using region-split experiments, with a forward-selection algorithm to optimize treated/control region matching. The result is a continuously calibrated causal engine for budget allocation and scenario planning.

Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces

Step 1: from decisions to negative user experiences

Get Iraklikhorguani ’s stories in your inbox

Step 2: from negative user experiences to future outcomes

Step 3: verify overall LTE by region-split experiments