Uber and OpenAI have each unveiled innovative infrastructure-level control platforms designed to enhance system performance and reliability under demanding conditions. Uber’s Global Rate Limiter is built to handle an extraordinary 80 million requests per second using a technique called probabilistic traffic shedding. This sophisticated method is integrated with a three-tier feedback loop that significantly reduces latency by eliminating dependencies on Redis, offering a more efficient and responsive system.
On a parallel front, OpenAI has introduced the Access Engine, which departs from traditional rate-limiting practices by implementing a ‘credit waterfall’ mechanism. This system is crafted to manage user access by transitioning seamlessly from conventional rate limits to a credit-based method. This approach is particularly beneficial in maintaining momentum for users engaged in creative workflows, ensuring that operations remain smooth and uninterrupted.
Both platforms showcase a commitment to advanced, distributed enforcement and the use of soft controls, which are instrumental in ensuring overall system stability at a vast scale. These developments highlight a shift from manual intervention towards automated, adaptive systems capable of managing complex, high-volume infrastructures.

Collections

Uber and OpenAI have each launched infrastructure-level control platforms targeting high-scale system reliability. Uber's Global Rate Limiter handles up to 80 million requests per second using probabilistic traffic shedding and a three-tier feedback loop, removing Redis dependencies to reduce latency. OpenAI's Access Engine replaces traditional rate limiting with a 'credit waterfall' mechanism that transitions users from conventional rate limits to a credit-based system, preserving continuity for creative workflows. Both platforms emphasize distributed enforcement, soft controls, and automated adaptive management at massive scale.

Uber and OpenAI Introduce Advanced Infrastructure-Level Control Platforms