Digital pinboard provider Pinterest has published an article explaining its blueprint for the future of large-scale data processing with its new platform Moka. The company is moving core workloads fro

InfoQ is a leading online platform for software developers, architects, and technical leaders, providing news, articles, presentations, and interviews on a wide range of topics, including agile practices, DevOps, microservices, and emerging technologies. With a focus on quality content and expert insights, InfoQ helps professionals stay informed about the latest trends, best practices, and industry developments. Developers can learn from real-world experiences, gain  knowledge, and connect with peers in the global software community through InfoQ's diverse and engaging content.

InfoQ

Pinterest has migrated its large-scale data processing infrastructure from Hadoop to a Kubernetes-based platform called Moka, running on Amazon EKS with Apache Spark as the primary engine. The platform addresses scalability, security, and cost requirements while supporting multiple processing frameworks including Flink and Ray. Key technical implementations include standardized logging with Fluent Bit, metrics via OpenTelemetry and Prometheus, infrastructure-as-code using Terraform and Helm, and multi-architecture support for Intel and ARM instances. The migration represents an industry shift toward treating Kubernetes as a control plane for data workloads, with Pinterest sharing their phased approach, observability investments, and lessons learned from operating Spark at massive scale.

Pinterest's Moka: How Kubernetes Is Rewriting the Rules of Big Data Processing