In November 2020, we introduced the concept of prioritized load shedding at the API gateway level in our blog post, Keeping Netflix Reliable Using Prioritized Load Shedding. Today, we’re excited to…

The Netflix Tech Blog provides insights into the engineering practices, architecture, and innovations behind the Netflix streaming platform. Developers can learn about Netflix's microservices architecture, data infrastructure, and machine learning algorithms, as well as explore topics such as video encoding, content delivery, and personalization algorithms used in the Netflix service.

Netflix TechBlog

Netflix has enhanced its reliability by extending prioritized load shedding techniques from the API gateway level to the individual service level, specifically for the video streaming control plane. This approach prioritizes critical user-initiated requests over non-critical pre-fetch requests, using partitioned concurrency limiters. This strategy proved effective during high-traffic incidents, ensuring high availability for user-initiated requests. Netflix also developed an internal library for prioritized load shedding using predefined priority buckets and incorporated CPU and IO-based load shedding techniques to maintain system performance under stress.

Enhancing Netflix Reliability with Service-Level Prioritized Load Shedding