I ran some very basic to seriously powerful AI models on the Raspberry Pi 5. The result in not very unsurprising.

ItsFOSS's platform is a central hub for Linux enthusiasts and open-source advocates, offering insights into Linux distributions, software applications, and Linux-related news. Through articles, tutorials, and reviews, ItsFOSS offers insights into using Linux as a desktop operating system, developing open-source software, and contributing to the Linux community. Readers can learn about Linux tips and tricks, open-source alternatives to proprietary software, and the latest developments in the Linux ecosystem.

It's Foss

The Raspberry Pi 5, equipped with a 4-core Cortex-A76 CPU, up to 8GB of RAM, and a VideoCore VI GPU, was used to test various large language models (LLMs) for their efficiency and performance. Models tested include Phi-3.5B, Gemma2-2B, Qwen2.5-3B, Mistral-7B, and Llama 2-7B, among others. Key metrics were inference time, accuracy, and resource utilization. Notably, models under 7 billion parameters generally performed well, with specific strengths found in different LLMs such as Qwen2.5's speed and Gemma2's efficiency. The results highlight the Pi's capability to handle AI tasks given the proper model selection.

I Ran 9 Popular LLMs on Raspberry Pi 5; Here's What I Found

<p>I tried to run large LLM for image generation on my PC and it took ages! I guess we still need to find good way to optimise those, but to think that actually it’s quite something that we can already use it on rasbery pi. Thank you for sharing.</p>


<p>Small correction, the Pi 5 had Videocore VII not VI.</p>
