ByteShape's device-optimized release of Qwen3-30B-A3B-Instruct-2507. Learn how we maximize speed and quality tradeoffs across Raspberry Pi, Intel CPUs, and NVIDIA GPUs.

Hacker News is a community-driven platform for sharing and discussing technology news, startups, and programming-related topics. Through user submissions and comments, Hacker News offers insights into emerging technology trends, industry developments, and entrepreneurial ventures. Readers can participate in discussions, share their insights, and stay informed about the latest advancements in technology and innovation.

Hacker News

ByteShape demonstrates running a 30B parameter Qwen3 model on resource-constrained devices like Raspberry Pi 5 at 8+ tokens per second using their Shapelearn bitlength learning method. The approach optimizes weight datatypes to maximize the TPS-quality tradeoff rather than just minimizing model size. Benchmarks across CPUs (Raspberry Pi, Intel i7) and GPUs (RTX 5090, RTX 4080) show ByteShape models consistently outperform Unsloth and MagicQuant alternatives, achieving higher throughput at equivalent quality or better accuracy at the same speed. The analysis reveals that GPU performance depends heavily on kernel choice and quantization format, with 4-bit quantizations often hitting sweet spots where lower bit-widths actually reduce speed due to memory access patterns and decode overhead.

A 30B Qwen Model Walks Into a Raspberry Pi… and Runs in Real Time