At Asana, our invalidation pipeline is a key part of implementing near-realtime reactivity in the webapp. Learn how the platform engineering team scaled the pipeline.

AsanaEng's platform is a central hub for engineering teams and project managers, offering insights into agile project management, collaboration tools, and productivity techniques. Through articles, case studies, and best practices guides, AsanaEng offers insights into using Asana for project planning, task tracking, and team communication. Readers can learn about agile methodologies, sprint planning, and Kanban boards to streamline their workflows and achieve project success.

Asana Engineering

Asana's platform engineering team tackled total load scalability and operational complexity in their invalidation pipeline. They addressed invalidation fanout issues through multiple approaches: attempting domain-isolated invalidations (which failed due to batching overhead), implementing invalidator replication per database as a stopgap, and ultimately migrating to a cell-based architecture for 10-20x fanout reduction. To reduce operational complexity from cache thrashing and binlog tailer issues, they're splitting the invalidator into separate generation and distribution components connected by an external datastore like Redis Streams or Kafka. Key lessons include biasing toward simplicity, validating core principles early, and recognizing the silent costs of maintaining complex generic systems.

Scaling our invalidation pipeline: Part 2 • Asana