Building a Distributed Persistent Queue That Scaled AI Workloads 5x

Salesforce's Agentforce team built a distributed persistent queue to orchestrate AI agent and human seller workflows under shared LLM infrastructure limits (300 RPM). The system uses a fair-share round-robin dispatcher, a three-tier priority queue (reply > intro > nudge emails), adaptive rate modulation with proactive admission control, and a dual-path architecture separating autonomous from human-review workflows. These design choices eliminated cascading rate-limit failures, prevented workload starvation, and achieved a 5x throughput increase, enabling customers to process over 10,000 leads daily.