How Meta Used AI to Map Tribal Knowledge in Large-Scale Data Pipelines

Meta's engineering team tackled a core limitation of AI coding assistants: they lack context about proprietary codebases. For a large-scale data pipeline spanning 4,100+ files across four repos and three languages, they built a pre-compute engine using 50+ specialized AI agents that systematically read every file and produced 59 concise context files encoding tribal knowledge. The result was 100% AI context coverage (up from 5%), documentation of 50+ non-obvious patterns, and roughly 40% fewer AI agent tool calls per task. The system is model-agnostic and self-maintaining, with periodic automated validation and self-repair. The post also outlines a reusable five-question framework and 'compass, not encyclopedia' design principle applicable to any large proprietary codebase.