I Ran 9 Popular LLMs on Raspberry Pi 5; Here's What I Found

The Raspberry Pi 5, equipped with a 4-core Cortex-A76 CPU, up to 8GB of RAM, and a VideoCore VI GPU, was used to test various large language models (LLMs) for their efficiency and performance. Models tested include Phi-3.5B, Gemma2-2B, Qwen2.5-3B, Mistral-7B, and Llama 2-7B, among others. Key metrics were inference time, accuracy, and resource utilization. Notably, models under 7 billion parameters generally performed well, with specific strengths found in different LLMs such as Qwen2.5's speed and Gemma2's efficiency. The results highlight the Pi's capability to handle AI tasks given the proper model selection.