How DoorDash optimized its in-house search engine - taming latency spikes and uncovering deep JVM-level insights for massive performance gains.

DoorDash offers insights into food delivery technology, logistics, and customer experience. Developers can delve into DoorDash's engineering blogs, tech talks, and innovation stories to gain a deeper understanding of how technology powers the food delivery industry. By exploring topics such as scaling infrastructure, optimizing delivery routes, and enhancing user experience through innovative technology solutions, developers can use insights to apply to their own projects and industries.

Doordash

DoorDash's search platform team investigated CPU utilization variance and latency spikes in their searcher pods. Through deep profiling using perf stat and flame graphs, they discovered that Shenandoah GC's memory barriers in JIT-compiled code were causing significant overhead in hot Lucene functions. Switching to G1GC and upgrading to Lucene 10.2 resulted in 30% improved p99.9 latency, 35-40% reduced CPU utilization, eliminated pod performance variance, and delivered an additional 30% reduction in hardware costs across their entire search platform.

Optimizing DoorDash’s in-house search engine platform

The problem: High variance in Searcher CPU utilization