What Breaks at 1M AI Requests per Day?

At 1M AI inference requests per day (~11.6 req/s average, but far higher at peak), the model itself rarely breaks first. The real failure points are the surrounding serving infrastructure: queueing latency that hides inside average metrics, GPU contention where high utilization doesn't mean high throughput, cost explosions driven by token volume and idle GPU time, autoscaling lag triggered by wrong signals (CPU instead of queue depth or GPU memory), and batching trade-offs that hurt tail latency. Key recommendations include tracking p95/p99 latency and time-to-first-token, monitoring queue depth separately from execution time, scaling on inference-specific signals, classifying requests into priority tiers, right-sizing models per task, and defining explicit SLOs for latency and cost per token.