A step-by-step guide to running PyFlink jobs on Kubernetes using Apache Flink 1.18. Covers setting up a local Kubernetes cluster with kind, installing the Flink Kubernetes Operator via Helm, deploying a single-node Kafka cluster using the Strimzi operator, writing a simple PyFlink job that generates random orders with the DataGen connector and writes them to Kafka via Flink SQL, building a custom Docker base image to work around a JDK/JRE dependency issue with PemJa, and deploying the job using a FlinkDeployment custom resource.
Sort: