Jeff Geerling is a software developer, author, and open-source contributor known for his expertise in infrastructure automation and Drupal development. Readers can learn about DevOps practices, Ansible automation, and web development with Drupal. With blog posts, tutorials, and open-source projects, Jeff Geerling shares  insights and best practices for building and managing web infrastructure.

Jeff Geerling

Testing RDMA over Thunderbolt 5 on a four-Mac Studio cluster with 1.5 TB unified memory shows significant performance gains for running massive AI models. The M3 Ultra Mac Studio outperforms comparable systems from Nvidia and AMD in CPU, AI inference, and power efficiency benchmarks. RDMA support in Exo 1.0 enables linear performance scaling across nodes, achieving 30+ tokens/second on trillion-parameter models. However, limitations include Thunderbolt 5's four-node maximum, macOS cluster management challenges, stability issues with prerelease software, and lack of standard networking options like QSFP for larger deployments.

1.5 TB of VRAM on Mac Studio - RDMA over Thunderbolt 5

Unanswered Questions / Topics to Explore Further

<p>well that’s apple M series chips for you</p>
