Google has released Android Bench, an official leaderboard benchmarking LLMs on real-world Android development tasks. The benchmark uses tasks sourced from public GitHub Android repositories, covering areas like resolving breaking changes, wearable networking, and Jetpack Compose migrations. Models are evaluated by having them fix reported issues, verified via unit or instrumentation tests. In the initial release, models completed 16–72% of tasks, with Gemini 3.1 Pro scoring highest, followed by Claude Opus 4.6. The methodology, dataset, and test harness are publicly available on GitHub. Google aims to help LLM makers identify gaps and improve Android-specific capabilities, ultimately benefiting developers using AI assistance in Android Studio.
Sort: