New LLM optimization technique slashes memory costs up to 75%

Researchers at Sakana AI have developed 'Universal Transformer Memory,' a technique using Neural Attention Memory Models (NAMMs) to optimize language models by keeping pertinent information and discarding redundant data. This innovation helps enterprises reduce the cost and improve the efficiency of applications built on large language models. NAMMs are trained separately and can be applied to various models, enhancing their performance on different tasks while saving up to 75% of cache memory. The technique is particularly useful for enterprises dealing with large amounts of data.