Best of Data Streaming — March 2025

1. 1
   Article

   The New Stack·1y

   The New Look and Feel of Apache Kafka 4.0

   Apache Kafka 4.0 introduces significant upgrades, including the replacement of ZooKeeper with KRaft for metadata management, enhancing stability and reducing complexity. The release features Queues for Kafka to allow scaling consumers beyond topic partitions, improved consumer group rebalancing, and new capabilities for code injection and observability. These updates aim to streamline Kafka's operations and improve the developer experience.

   32
2. 2
   Article

   SingleStore·1y

   Can a Database Be Faster Than a Formula 1 Engine?

   Formula 1 cars generate vast amounts of real-time telemetry data, which is crucial for strategic decision-making. Each car is fitted with 300 sensors producing 1.1 million data points per second. Teams use this data for simulations, performance analysis, and strategy adjustments. SingleStore's real-time analytics capabilities are highlighted, showcasing its ability to handle high-throughput data streams and provide millisecond response times. The post includes a practical guide for setting up a data ingestion and visualization simulation using SingleStore, Confluent Kafka, and Grafana.

   25

   1
3. 3
   Article

   Data Engineer Things·1y

   Bufstream: Stream Kafka Messages to Iceberg Tables in Minutes

   Bufstream offers a cost-effective and cloud-native alternative to Kafka by using object storage, significantly reducing infrastructure costs. It enhances data quality management by integrating schema validation directly into the broker and seamlessly transforms Kafka messages into Iceberg tables, simplifying the data pipeline. Bufstream also addresses challenges with Kafka's cloud inefficiencies and provides built-in support for schema enforcement and granular access control.

   25
4. 4
   Article

   Dev Genius·1y

   Change Data Capture Tools

   Change Data Capture (CDC) tools automatically track and replicate database changes in real time. Different mechanisms like log-based, trigger-based, query-based, timestamp-based, and hybrid CDC tools are used. Debezium is a popular open-source CDC platform with high scalability, integrating well with Kafka. Other tools like DBConvert Streams, Maxwell Daemon, and Sequin offer various features and integrations for efficient data replication. Challenges such as setup complexity and performance overhead are common with these tools.

   22