Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces

Lyft's Foundational Models team describes a methodology for estimating long-term, market-mediated effects of pricing and incentive decisions in a two-sided marketplace. Standard A/B tests can't capture these effects because policy changes ripple through supply and demand in complex ways. The framework uses a two-step surrogacy approach: first, residualized regression models map policy changes to shifts in negative user experiences (wait times, surge, driver idleness); second, doubly-robust AIPW causal estimation maps those experience shifts to future behavioral outcomes via a surrogacy index. Both steps are validated with switch-back and user-split experiments respectively. Finally, direct and market-mediated long-term effects are combined and verified end-to-end using region-split experiments, with a forward-selection algorithm to optimize treated/control region matching. The result is a continuously calibrated causal engine for budget allocation and scenario planning.