Supermetal's new Iceberg sink snapshots Postgres data in 13 minutes versus 90+ for Flink, Kafka Connect, and Spark in this CDC benchmark.

The New Stack is a publication covering trends and technologies in cloud-native development, DevOps, and software delivery. Developers can learn about containerization, Kubernetes, and cloud computing, as well as explore topics such as microservices architecture, serverless computing, and continuous integration/continuous delivery (CI/CD) pipelines.

The New Stack

A sponsored benchmark comparing Supermetal's new Iceberg sink against Apache Flink, Kafka Connect (Debezium), and Apache Spark for CDC-based Postgres-to-Iceberg pipelines. Using TPC-H SF=50 data on identical single-node AWS infrastructure, Supermetal completed snapshotting in 13 minutes with no tuning, while Flink took 90–116 minutes, Kafka Connect 120 minutes, and Spark over 3 hours. The key differentiators are Supermetal's fast CDC source, low serialization overhead, and its unique ability to switch Iceberg sink behavior (append-only with target file size vs. merge-on-read with time-based flush) between snapshot and live CDC phases. Flink required aggressive fetch/split size tuning; Kafka Connect needed careful batch tuning; Spark struggled on single-node due to its scale-out architecture. All tools produced correct data with matching row counts.

Postgres to Iceberg in 13 minutes: How Supermetal compares to Flink, Kafka Connect, and Spark