Mintlify replaced expensive sandbox environments with ChromaFs, a virtual filesystem built on top of their existing Chroma vector database, to power their documentation AI assistant. The sandbox approach had a ~46-second p90 boot time and would cost over $70,000/year at scale. ChromaFs intercepts UNIX shell commands (grep, cat, ls, find) and translates them into Chroma queries, reducing session creation to ~100ms with near-zero marginal cost. Key implementation details include: bootstrapping a gzipped directory tree stored in Chroma, per-user access control via metadata filtering, lazy file pointers for large OpenAPI specs in S3, a two-phase grep optimization using Chroma as a coarse filter and Redis caching, and a read-only design that keeps sessions stateless.
Sort: