A stable patch for the `amdgpu` Linux kernel driver now allows AMD RX series GPUs (400 to 7000) to work with Raspberry Pi 5, supporting both Vulkan graphics and compute API. This guide provides steps to install `llama.cpp` with Vulkan support on the Pi 5. While larger models face performance issues due to inefficient memory access translations by the `amdgpu` driver, smaller models perform well. The compact, power-efficient setup only uses about 10-12W when idle, making it an attractive option for local machine learning tasks.

5m read timeFrom jeffgeerling.com
Post cover image
Table of contents
Installing llama.cpp with Vulkan support on the Pi 5PerformanceGoals

Sort: