Helix adapted Moonlight, a gaming streaming protocol designed for single-player sessions, to stream GPU-accelerated desktop environments for AI agents to multiple users simultaneously. The team initially used "apps mode" with a workaround where their API pretended to be a client to start containers, but are migrating to "lobbies mode" which natively supports multi-user access to shared sessions. The solution enables low-latency (50-100ms) streaming of full Linux desktops with AI agents working in real IDEs and browsers, though challenges remain with input scaling and video corruption across different client resolutions.
Table of contents
Why Stream Desktops for AI Agents?Enter Moonlight (and Wolf)The Protocol MismatchApps Mode: Our First WorkaroundLobbies Mode: The Real SolutionThe Current Reality (And Remaining Bugs)What This Looks Like In PracticeWhy This MattersWhat We LearnedWhat’s Next1 Comment
Sort: