The Smokejumpers: Scaling Gemini’s serving infrastructure

Google Fellow Emanuel Taropa discusses the infrastructure challenges of scaling Gemini models to billions of users. The Smokejumpers team handles model serving, capacity allocation, and optimization across Google's TPU infrastructure. Key challenges include balancing cache hit rates, managing capacity constraints across model Show more