Apache Kafka is a very popular distributed event streaming system. It's most popular use cases are message sending and event processing.

In this video, we dive into what Apache Kafka is, how it helps engineers scale their messaging systems, and how it works internally. We will be referencing the 2011 paper by Jay Kreps for this video.

Common design patterns with Kafka include publisher-subscriber and event sourcing. If you are new to Kafka, or looking to understand its internals, this video will be useful to you!

00:00 What is Apache Kafka?
01:47 High-Level Design
04:36 Scaling Kafka
07:45 Message Batching
08:40 Atleast Once delivery
09:40 Atmost Once delivery
10:07 Exactly Once Delivery
12:45 Zero Copy Messaging
15:12 Thank you!

Garbage Collection (Nepotism): https://www.youtube.com/watch?v=ZhbIReLe-r8
Research Paper: https://notes.stephenholiday.com/Kafka.pdf
System Design Course: https://interviewready.io/
Designing Data-Intensive Applications Book: https://amzn.to/3SyNAOy

#Kafka #ResearchPaper #SytemDesign

YouTube

Apache Kafka, developed by LinkedIn in 2011, is an open-source messaging system known for its scalability and reliability. Kafka is primarily used for event streaming and message queuing, making it ideal for systems requiring high throughput and fault tolerance. It includes key components like producers, brokers, and consumers, and ensures data consistency with features like partitions, replicas, and the at-least-once delivery guarantee. Kafka also optimizes performance with techniques like zero-copying which enhance message throughput significantly.

Apache Kafka in 15 minutes

<p>Thanks for sharing. Please upload some more.</p>
