At Asana, our invalidation pipeline is a key part of implementing near-realtime reactivity in the webapp. Learn how the platform engineering team scaled the pipeline.

AsanaEng's platform is a central hub for engineering teams and project managers, offering insights into agile project management, collaboration tools, and productivity techniques. Through articles, case studies, and best practices guides, AsanaEng offers insights into using Asana for project planning, task tracking, and team communication. Readers can learn about agile methodologies, sprint planning, and Kanban boards to streamline their workflows and achieve project success.

Asana Engineering

Asana's platform engineering team scaled their invalidation pipeline to handle near-realtime reactivity for millions of users. The pipeline uses change-data-capture from MySQL binlog to decouple mutations from invalidations, automatically determining affected data and notifying clients. The first major scaling challenge was expensive transactions causing head-of-line blocking. The solution involved introducing out-of-order invalidation delivery with a DataFreshness construct (similar to vector clocks) that maintains eventual consistency while allowing slow transactions to be processed asynchronously. This approach eliminated manual intervention and has worked successfully for 6 years without further optimization.

Scaling our invalidation pipeline: Part 1 • Asana

How did we actually solve these problems?

Problem 1: Expensive transactions and head-of-line blocking