High Cardinality Metrics: How Prometheus and ClickHouse Handle Scale

Prometheus and ClickHouse handle high-cardinality metrics through fundamentally different architectures. Prometheus pays cardinality costs at write time through memory overhead, index maintenance, and series creation (~3-4KB per active series), leading to OOM failures during ingestion. ClickHouse defers costs to query time, storing data cheaply in columnar format but consuming memory during GROUP BY aggregations on high-cardinality dimensions. Neither system "solves" cardinality—they fail at different points. The article provides deep technical analysis of both systems' internals, including Prometheus's head block management, Gorilla compression, posting list indexes, and ClickHouse's sparse indexing, ORDER BY optimization strategies, and vectorized execution. It explains why sharding Prometheus doesn't eliminate the problem, how native histograms reduce cardinality 10-20x, and why hybrid pipelines using streaming aggregation often make sense. The key insight: design decisions about high-cardinality labels should consider which failure mode is acceptable for your use case.