Cogent Security built an AI-native vulnerability management platform on ClickHouse, migrating from Postgres to achieve sub-second query latency on hundreds of millions of rows without caching. The architecture uses a hot/cold data lakehouse with Iceberg on S3 and ClickHouse as the serving layer, with denormalized tables, projections, and tuned compression codecs as core design principles. Their Chart Agent uses an agentic loop with RAG, live SQL execution, and interleaved thinking to convert natural language into ClickHouse SQL, improving accuracy from 40% to 94%. Pre-computed ClickHouse projections were the single biggest accuracy driver, simplifying the SQL the agent needed to generate. An in-house Ontology Service keeps the data model and its semantics co-located to prevent agent confusion when schemas change or customers have custom fields.
Table of contents
The growing threat gap #Security data at scale #Designing for speed #Sub-second performance #Making reporting conversational #Iterating toward accuracy #One source of truth #A virtuous cycle #Sort: