A customer query was taking over 4× longer than it should have. The profiler pointed at Rust code we'd assumed was free. We followed the trail all the way down to assembly to find the true cost.

Lobsters is a community-driven platform for sharing and discussing links to articles, tutorials, and projects related to technology and programming. Readers can learn about a wide range of topics, from software development and system administration to cybersecurity and artificial intelligence. With user submissions, comments, and voting, Lobsters provides a platform for collaborative learning and knowledge sharing among technology enthusiasts.

Lobsters

A full-text search query on turbopuffer was taking 220ms instead of the expected ~50ms. Profiling revealed that over 60% of runtime was spent in a merge iterator, not in BM25 ranking. The root cause: Rust's zero-cost iterator abstraction, while compiling each individual call efficiently, prevented the compiler from vectorizing or unrolling across calls due to the recursive nature of `next()`. The fix was a classic database technique — batched iterators — where `next_batch()` fills an array of 512 KV pairs at once, giving the compiler a tight inner loop it can auto-vectorize with SIMD. The result: the benchmark dropped from 6.5ms to 110μs (60× faster), and the production query latency fell from 220ms to 47ms. The key lesson: 'zero-cost' means the abstraction compiles away per call, not that it has no effect on the compiler's ability to optimize across calls.

Rust zero-cost abstractions vs. SIMD

Breaking the abstraction to find a solution

<p>can Rustaceans willing to fix this zero cost abstraction optimization problem?</p>
