A detailed walkthrough of converting the dots.ocr model (a 3B parameter OCR model from RedNote) to run on Apple devices using Core ML and MLX. The guide covers the conversion process from PyTorch to Core ML, including simplifying the model architecture, debugging common conversion errors, and initial benchmarking. Key challenges addressed include handling attention implementations, fixing dtype mismatches, removing dynamic control flow, and dealing with variable-length sequence masking. The converted model initially runs on GPU in FLOAT32 precision, with future parts promising Neural Engine optimization and quantization techniques.

8m read timeFrom huggingface.co
Post cover image
Table of contents
ConversionDots.OCRStep 0: Understand and simplify the modelStep 1: A simple harnessStep 2: Bug huntingStep 3: Benchmarking

Sort: