How Netflix Built a Real-Time Distributed Graph for Internet Scale

Netflix built a Real-Time Distributed Graph (RDG) to track member interactions across streaming, gaming, and other services. The system processes millions of events per second using Apache Kafka for ingestion, Apache Flink for stream processing, and a custom Key-Value Data Abstraction Layer (KVDAL) built on Cassandra for storage. Netflix rejected traditional graph databases like Neo4j and AWS Neptune due to scalability limitations and operational complexity, instead emulating graph capabilities using key-value storage with adjacency lists. The architecture handles 8 billion nodes, 150 billion edges, 2 million reads/second, and 6 million writes/second across 2,400 EC2 instances, with each node and edge type isolated in separate namespaces for independent scaling.