Scale-to-Zero Without Serverless Platforms

Scaling an application to zero replicas when idle is trivial, but the real challenge is handling incoming requests during cold starts without dropping them. The key insight is that requests must be held (queued) rather than dropped when no replicas are running, requiring a proxy or queue layer in front of the application to buffer traffic until instances come back up.