A workshop covering how to apply DevOps and CI/CD practices to Apache Spark applications running on containerized platforms. The session demonstrates packaging Spark jobs as immutable artifacts, implementing automated quality gates including code and data tests, and promoting jobs through environments using pipeline-as-code. Key topics include the complete flow from commit to deployment, production considerations for platform teams including multi-tenant quotas and security, and practical templates for implementation.
1 Comment
Sort: