Airbnb completely rearchitected their key-value store Mussel from v1 to v2, migrating from a custom storage backend to a NewSQL system. The new architecture addresses operational complexity, capacity issues, and consistency limitations while maintaining sub-25ms p99 read latency. The migration involved a blue-green strategy with dual writes, custom pipelines, and zero-downtime transfer of over a petabyte of data across thousands of tables. Mussel v2 now combines bulk ingestion capabilities with real-time streaming and provides better cost visibility through Kubernetes-native operations.

10m read timeFrom medium.com
Post cover image
Table of contents
Why rearchitectKey Challenges with v1New architectureDispatcherTTLThe migration processChallengeProcessGet Shravan Gaonkar’s stories in your inboxMigration pipelineLessons learnedConclusion and next stepsReferences

Sort: