How we bypassed CoreML and talked directly to the hardware

Hacker News is a community-driven platform for sharing and discussing technology news, startups, and programming-related topics. Through user submissions and comments, Hacker News offers insights into emerging technology trends, industry developments, and entrepreneurial ventures. Readers can participate in discussions, share their insights, and stay informed about the latest advancements in technology and innovation.

Hacker News

A deep technical reverse engineering of Apple's M4 Neural Engine (ANE), bypassing CoreML to talk directly to the hardware. The authors discovered 40+ private classes in AppleNeuralEngine.framework, cracked the MIL (Machine Learning Intermediate Language) compilation path and E5 binary format, and achieved direct ANE access via the private _ANEClient API without CoreML. Key findings include: the ANE is a graph execution engine with 16 cores and a queue depth of 127, E5 binaries are tiny parameterized programs (~2.7KB regardless of matrix size), IOSurfaces enable zero-copy GPU↔ANE transfers, and an in-memory compilation path exists via _ANEInMemoryModelDescriptor. The series continues with benchmarking (Part 2) and training on ANE (Part 3), with all code published on GitHub.

Inside the M4 Apple Neural Engine, Part 1: Reverse Engineering

_ANEClient: Direct Access to the Neural Engine