CMU Researchers Unveil RoboTool: An AI System that Accepts Natural Language Instructions and Outputs Executable Code for Controlling Robots in both Simulated and Real-World Environments

CMU researchers and Google DeepMind have developed RoboTool, an AI system that allows robots to use tools creatively, solve complex tasks, and handle implicit physical constraints. The system leverages Large Language Models (LLMs) and consists of an analyzer, planner, calculator, and coder. RoboTool outperforms traditional methods and addresses the challenge of creative tool use in robots.