Building a Next-Generation Key-Value Store at Airbnb How we completely rearchitected Mussel, our storage engine for derived data, and lessons learned from the migration from Mussel V1 to V2. By …

Airbnb's platform is  dedicated to providing insights and resources for developers and technology enthusiasts, focusing on engineering challenges, innovation projects, and best practices in software development. Through articles, tech talks, and open-source contributions, Airbnb offers insights into building scalable and reliable technology solutions. Developers can learn about Airbnb's engineering culture, technology stack, and contributions to the broader tech community.

Airbnb

Airbnb completely rearchitected their key-value store Mussel from v1 to v2, migrating from a custom storage backend to a NewSQL system. The new architecture addresses operational complexity, capacity issues, and consistency limitations while maintaining sub-25ms p99 read latency. The migration involved a blue-green strategy with dual writes, custom pipelines, and zero-downtime transfer of over a petabyte of data across thousands of tables. Mussel v2 now combines bulk ingestion capabilities with real-time streaming and provides better cost visibility through Kubernetes-native operations.

Building a Next-Generation Key-Value Store at Airbnb

Get Shravan Gaonkar’s stories in your inbox