As promised in the last article, we will continue learning Apache Kafka this week. In this article, I will present my research on some of Kafka’s important designs: Filesystem, Zero-copy, and…

Samuel Maudo

Couchbase

Community Picks is a section on daily.dev where our community members share the most interesting and valuable content they've discovered online. From insightful articles to handy tools, every post is a gem curated by our dedicated coomunity. To contribute to Community Picks, you need to have at least 250 reputation points, ensuring that only active and trusted members can share their finds.

Community Picks

Apache Kafka leverages the OS filesystem for data storage, using the page cache to improve performance without adding memory overhead from Java objects. Kafka employs sequential access patterns to optimize read/write operations, benefiting from zero-copy optimization by reducing context switches and data transfers between user and kernel spaces. Batching messages enhances network and disk operation efficiency, ensuring high performance.

Apache Kafka — Important Designs. Filesystem, Zero-copy, and Batching