Best of Kafka — September 2025

  1. 1
    Article
    Avatar of netflixNetflix TechBlog·34w

    Building a Resilient Data Platform with Write-Ahead Log at Netflix

    Netflix built a generic Write-Ahead Log (WAL) system to solve data consistency and reliability challenges at scale. The system provides a simple API that abstracts underlying message queues (Kafka, SQS) and supports multiple use cases including delayed queues, cross-region replication, and multi-partition mutations. WAL prevents data loss, handles system entropy across different datastores, and enables reliable retry mechanisms for real-time data pipelines. The architecture separates message producers from consumers, uses configurable namespaces for logical separation, and leverages Netflix's Data Gateway infrastructure for deployment. Key applications include EVCache cross-region replication, Live Origin's delayed delete operations, and Key-Value service's MutateItems API with two-phase commit semantics.

  2. 2
    Article
    Avatar of bytebytegoByteByteGo·37w

    How Netflix Tudum Supports 20 Million Users With CQRS

    Netflix redesigned their Tudum platform architecture to support 20 million users by replacing a traditional CQRS implementation with RAW Hollow, an in-memory object store. The original design used Kafka and Cassandra with caching layers, causing delays in editorial previews due to eventual consistency. By embedding RAW Hollow directly into microservices, they eliminated external datastores and reduced page construction time from 1.4 seconds to 0.4 seconds while enabling near-instant editorial previews. The compressed in-memory approach stores three years of data in just 130MB while maintaining strong consistency options for critical workflows.

  3. 3
    Article
    Avatar of awegoAwesome Go·35w

    Watermill Quickstart

    Watermill has released an interactive quickstart tutorial for learning their Go library for building event-driven applications. The hands-on training covers core APIs (Publisher/Subscriber, Router, CQRS), Kafka integration, topic topology, middleware, consumer groups, message ordering, the Outbox pattern with PostgreSQL, and switching Pub/Sub implementations with Redis. The tutorial runs directly in your IDE without requiring Docker setup or browser-based coding.

  4. 4
    Article
    Avatar of confConfluent Blog·37w

    Apache Kafka 4.1 Release: New Features & Upgrade Guide

    Apache Kafka 4.1.0 introduces Queues in preview, a new Streams Rebalance Protocol in early access, and OAuth jwt-bearer grant type support. Key improvements include enhanced consumer group handling, better transaction error handling, plugin metrics registration, and Connect support for multiple connector versions. The release focuses on making Kafka more robust and easier to manage in production environments.

See all Kafka archives