How we solved request logging at scale by moving from MariaDB to ClickHouse, Kafka, and Vector after our deprecated database engine couldn't keep up with billions of monthly requests.

Hacker News is a community-driven platform for sharing and discussing technology news, startups, and programming-related topics. Through user submissions and comments, Hacker News offers insights into emerging technology trends, industry developments, and entrepreneurial ventures. Readers can participate in discussions, share their insights, and stay informed about the latest advancements in technology and innovation.

Hacker News

Geocodio migrated their request logging system from MariaDB with deprecated TokuDB to ClickHouse Cloud after hitting performance issues at billions of monthly requests. The solution involved introducing Kafka for event streaming and Vector for batch processing, moving from individual row inserts to batched inserts of 30k-50k records. The migration strategy used feature flags to run both systems in parallel, enabling zero-downtime validation before fully switching over.

From Millions to Billions

Attempt #1: Just Swap MariaDB for ClickHouse

The Solution: Kafka + Vector + ClickHouse

<p>Good read! I’m curious about the choice to use Vector in between, why not use the built-in ClickHouse Kafka Connect Sink, which can also be configured for batch inserts?</p>
