Google introduces Transformer 2.0, focusing on enhanced long-term memory through Titans, a hybrid model combining attention mechanisms and a new neural memory module. Researchers also explore compute-optimal sampling for training language models, favoring weaker but cheaper models. An open-source model, Sky-T1-32B-Preview, Show more