Sprig, an AI-powered product research platform tracking 15 billion visitors and 1.3 trillion events, outgrew PostgreSQL, then a ClickHouse+Redis hybrid setup. Both systems hit P99 latencies around 50ms and faced scaling walls. After evaluating DynamoDB, DataStax Astra, Cassandra, and self-hosted ScyllaDB, the team chose ScyllaDB Cloud for its low latency, native Materialized Views, and managed operations. Migration used a dual-write Kafka connector plus historical backfill, with a validation layer comparing both systems until 99.99% agreement. The result: mean read latencies of ~500 microseconds and P90 of 1-2ms — 4-8x better than Redis — with reduced operational complexity and easier horizontal scaling.
Table of contents
The complexity behind Sprig’s surface-level simplicityPhase 1: Just use PostgresPhase 2: Add ClickHouse and RedisWhere Postgres + Redis + Clickhouse fell shortFinding the right database fitPhase 3: Moving to ScyllaDB CloudSort: