Microsoft AI Research Introduces SIGMA: An Open-Source Research Platform to Enable Research and Innovation at the Intersection of Mixed Reality and AI

Recent breakthroughs in generative AI and huge language, vision, and multimodal models can be a foundation for open-domain knowledge, inference, and generation capabilities, enabling open-ended task aid scenarios. Microsoft AI Research introduces SIGMA, an open-source research platform that combines mixed-reality and artificial intelligence technologies. SIGMA uses HoloLens 2 to walk users through procedural tasks and can provide answers to open-ended questions using extensive language models. It also highlights task-relevant objects in the user's field of view. SIGMA is built on the Platform for Situated Intelligence (psi) architecture. The researchers hope that by making SIGMA publicly available, it will facilitate future research in the convergence of mixed reality and AI.