OS-Genesis: A Novel GUI Data Synthesis Pipeline that Reverses the Conventional Trajectory Collection Process

OS-Genesis is an innovative data synthesis framework designed to enhance GUI agent training by reversing traditional task-oriented data collection methods. Instead of relying on expensive human annotation or predetermined tasks, it uses interaction-driven reverse task synthesis to generate diverse and high-quality training data autonomously. By incorporating exploratory interactions and retrospective analysis, it transforms these into coherent and comprehensive high-level tasks. Validated on benchmarks like AndroidWorld and WebArena, OS-Genesis has significantly improved task planning and execution, demonstrating its potential to advance digital automation and AI research.