AI Edge Torch: High Performance Inference of Pytorch Models on Mobile Devices

Google AI Edge Torch is a new offering that provides a direct path from PyTorch to the TensorFlow Lite runtime for high-performance inference of PyTorch models on mobile devices. It offers easy conversion, great model coverage, and CPU performance. The release includes coverage and performance improvements over existing workflows, early adoption by Shopify for on-device background removal, and partnerships with hardware companies for improved performance and coverage. Qualcomm has also released a new TensorFlow Lite delegate that provides significant speedups using DSP and neural processing units.