Titans by Google: The Era of AI After Transformers?

Google Research's Titans paper introduces a new model architecture designed to overcome the quadratic scaling limitation of Transformers. Titans incorporate a deep neural long-term memory module inspired by human memory, which learns to memorize at test time using a surprise-based update mechanism with adaptive forgetting. Four architectural variants are proposed: Memory as a Context, Memory as a Gate, Memory as a Layer, and LMM (memory-only). Benchmarks show Titans outperform baseline models on language modeling, commonsense reasoning, and especially long-sequence tasks like needle-in-a-haystack and BABILong, with Memory as a Context achieving the best overall results among hybrid models.