1.8 Trillion Events Per Day with Kafka: How Agoda Handles it

Agoda manages 1.8 trillion daily events through Apache Kafka with strategies like 2-step logging architecture, splitting Kafka clusters by use case, developing robust auditing systems, and dynamic load balancing solutions. Their approach ensures resiliency, flexibility, and efficient resource utilization despite hardware heterogeneity and inconsistent message workloads. Key solutions include lag-aware producers and consumers that adapt based on real-time data, mitigating over-provisioning issues and ensuring balanced workloads.