Discover more about what's new at AWS with Announcing the Apache Spark upgrade agent for Amazon EMR

AWS' platform is a leading cloud computing platform, offering insights into cloud infrastructure, services, and solutions for developers, businesses, and IT professionals. Through articles, whitepapers, and documentation, AWS offers insights into cloud architecture, serverless computing, and machine learning on AWS. Developers and architects can learn about AWS services like EC2, S3, Lambda, and more to build scalable, secure, and cost-effective cloud applications.

AWS launched an Apache Spark upgrade agent that automates version migrations for Amazon EMR on EC2 and EMR Serverless, reducing upgrade timelines from months to weeks. The agent analyzes PySpark and Scala code, identifies API changes, suggests modifications through conversational interfaces, and validates data quality throughout the process. Currently supports upgrades from Spark 2.4 to 3.5 and is available in all AWS Regions with SageMaker Unified Studio support, accessible through IDE Spaces, Kiro CLI, or MCP-compatible IDEs.