Apache Flink® on Kubernetes

Airbnb transitioned its stream processing architecture from Apache Hadoop Yarn to Kubernetes. This migration improved developer velocity, job availability, and infrastructure costs. The current setup integrates Flink directly with Kubernetes, offering a better developer experience, secure secrets management, isolated environments, enhanced monitoring, and simpler service discovery. The future focus will involve improving job availability, enabling autoscaling, and utilizing the Flink Kubernetes Operator for streamlined operations.