When we started building sandboxes for AI agents at Helix, we wanted to give each agent their own desktop environments that we could stream interactively to users’ browsers.

HelixML

Helix adapted Moonlight, a gaming streaming protocol designed for single-player sessions, to stream GPU-accelerated desktop environments for AI agents to multiple users simultaneously. The team initially used "apps mode" with a workaround where their API pretended to be a client to start containers, but are migrating to "lobbies mode" which natively supports multi-user access to shared sessions. The solution enables low-latency (50-100ms) streaming of full Linux desktops with AI agents working in real IDEs and browsers, though challenges remain with input scaling and video corruption across different client resolutions.

Technical Deep Dive on Streaming AI Agent Desktop Sandboxes: When Gaming Protocols Meet Multi-User Access

<p>Hope people stick to old HLD question in HLD rounds</p>
