Why can’t LLMs just LEARN the context window?

A research paper called 'End-to-End Test-Time Training for Long Context' proposes storing context information directly into a transformer's MLP weights via gradient updates at inference time, rather than relying on full attention or KV cache. The approach combines sliding window attention for local context with test-time weight Show more