Best of Kafka — April 2025

  1. 1
    Article
    Avatar of confConfluent Blog·1y

    Designing Event-Driven, Multi-Agent AI Architectures w/Kafka and Flink

    Learn how to build an event-driven, multi-agent AI architecture to simplify meal planning. Using tools like Kafka, Flink, LangChain, and Claude, the system coordinates multiple AI agents—each with specialized tasks like planning child-friendly or adult meals—into a cohesive meal plan. The approach ensures real-time responsiveness, adaptability, and fault tolerance, making complex daily tasks more manageable.

  2. 2
    Article
    Avatar of theburningmonktheburningmonk.com·1y

    Event versioning strategies for event-driven architectures

    The post discusses various strategies for versioning event schemas in event-driven architectures, including adding versions in event names, event payloads, separate streams, and using schema registries. It evaluates the pros and cons of each method and suggests alternative approaches to avoid breaking changes. The author emphasizes the importance of supporting backward compatibility and recommends always adding new fields instead of breaking existing schemas.

  3. 3
    Article
    Avatar of bytebytegoByteByteGo·1y

    How Airbnb Powers Personalization With 1M Events Per Second

    Airbnb utilizes its User Signals Platform (USP) to handle real-time personalization effectively, processing over 1 million events per second. USP ingests and processes user actions with sub-second latency, stores both real-time and historical data, and provides a configurable interface for developers. Key features include the use of Flink for event-driven processing, an append-only data model for resilience, and a robust infrastructure with hot standby Task Managers for operational stability.

  4. 4
    Article
    Avatar of javascript_developersJavascript developers·1y

    Build a Scalable Video Transcoding System with Node.js, Kafka & AI (Open-Source!)

    Learn to create an open-source, real-time video transcoding system using Node.js and Express for API services, Kafka for event-driven processing, FFmpeg and Whisper AI for transcoding and subtitles, AWS S3 and Elasticsearch for storage and search, and WebSockets for real-time progress updates. This tutorial is ideal for Node.js developers interested in video processing, scalable backends, or event-driven systems.

See all Kafka archives