Technical Deep Dive on Streaming AI Agent Desktop Sandboxes: When Gaming Protocols Meet Multi-User Access
Helix adapted Moonlight, a gaming streaming protocol designed for single-player sessions, to stream GPU-accelerated desktop environments for AI agents to multiple users simultaneously. The team initially used "apps mode" with a workaround where their API pretended to be a client to start containers, but are migrating to "lobbies mode" which natively supports multi-user access to shared sessions. The solution enables low-latency (50-100ms) streaming of full Linux desktops with AI agents working in real IDEs and browsers, though challenges remain with input scaling and video corruption across different client resolutions.