Inside Look at Reddit's Metadata Store

Reddit built a unified media metadata store to consolidate scattered metadata from multiple systems into a single AWS Aurora Postgres database. The system handles 100K+ read requests per second with p50 latency of 2.6ms and p99 of 17ms. The migration used dual writes, backfill, and a Kafka-based Change Data Capture pipeline to detect and fix inconsistencies. For future scalability toward an estimated 50TB by 2030, Reddit implemented range-based partitioning using pg_partman and pg_cron, leveraging monotonically increasing post_id values to keep recent partition indexes cached in the buffer pool.