Nextdoor's engineering team evolved their database architecture through a series of deliberate scaling steps. Starting from a single PostgreSQL instance, they added PgBouncer for connection pooling to address the process-per-connection bottleneck. They then introduced primary-replica architecture with time-based dynamic routing to handle read-heavy traffic while maintaining read-your-own-writes consistency. A Valkey (Redis-compatible) look-aside cache with MessagePack serialization and Zstd compression was layered on top for speed. To prevent stale cache data, they implemented a versioning system using PostgreSQL triggers and atomic Lua scripts for compare-and-set updates. Finally, Debezium-based Change Data Capture provides a self-healing reconciliation mechanism. Sharding by neighborhood ID represents the final scaling tier for write-heavy growth.

12m read timeFrom blog.bytebytego.com
Post cover image
Table of contents
New Year, New Metrics: Evaluating AI Search in the Agentic Era (Sponsored)The Limits of the “Big Box”Dividing the Labor and the “Lag” ProblemWhy writing code isn’t the hard part anymore (Sponsored)The High-Speed LibraryVersioning and Atomic UpdatesCDC and ReconciliationShardingConclusion

Sort: