Figma shares their experience in horizontally sharding their Postgres database to achieve nearly infinite scalability. They faced challenges such as inefficient query execution, loss of reliability and consistency properties, and the need for custom solutions. Figma developed a complex architecture using logical and physical sharding, views, and a custom query engine. The process of horizontally sharding a database is explained in detail, including the use of feature flags and shadow reads to ensure a safe migration. The article also highlights the need for further advancements in schema updates, global unique ID generation, atomic cross-shard transactions, ORM compatibility, and automated shard operations.

36m watch time
1 Comment

Sort: