Datadog has overhauled its timeseries indexing service by adopting an inverted index strategy inspired by search engines. This new approach consistently indexes every ingested tag, simplifying ingestion and query paths. Additionally, the indexing service now employs intranode sharding to leverage parallel processing effectively and has been rewritten in Rust for better performance. These enhancements have led to a 20x improvement in handling higher cardinality metrics, a 99% reduction in query timeouts, and a 50% reduction in operational costs.
Sort: