Netflix Open Sources Maestro, a Next-Gen Data Workflow Engine
Netflix has released Maestro, an open-source workflow orchestrator under an Apache 2.0 license. Designed for scalability and extensibility, Maestro supports hundreds of thousands of workflows and millions of jobs daily. It integrates with various open-source technologies and handles tasks such as retries, queuing, and task distribution. Originally announced in 2022, Maestro was developed to address performance limitations of Netflix's previous system, Meson, and is optimized for large-scale data workflows in various formats.