Community Picks

A detailed comparison between Apache Kafka and RabbitMQ, focusing on latency, throughput, CPU and memory usage, and disk operations. The test reveals that RabbitMQ offers lower latency, while Kafka provides higher throughput and resilience. The comparison extends to RabbitMQ Streams, where Kafka again shows superior performance. The post provides a comprehensive analysis suited for selecting the right messaging broker based on specific needs.

Apache Kafka vs RabbitMQ Performance (Latency - Throughput - Saturation)

ByteByteGo

Uber's real-time data infrastructure processes petabytes of data daily, supporting features like customer incentives and fraud detection. The system relies on Apache Kafka for streaming data, Apache Flink for stream processing, and Apache Pinot for real-time OLAP. Key requirements include consistency, availability, data freshness, scalability, and cost efficiency. Customizations and tools like FlinkSQL and uReplicator enhance reliability and performance. This enables real-time decisions such as dynamic pricing and operational insights. Scalability strategies, including Active-Active and Active-Passive Kafka setups, ensure high availability and fault tolerance.

How Uber Manages Petabytes of Real-Time Data

An open-source distributed event streaming platform used for building real-time data pipelines and streaming applications. Readers can explore Kafka's features, including topics, partitions, producers, consumers, and Kafka Connect for integrating with external systems.

Best of Apache Kafka — October 2024