DeepSeek Cracked The O(L²) Attention Bottleneck
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
DeepSeek V3.2 introduces DeepSeek Sparse Attention (DSA), reducing attention complexity from O(L²) to O(Lk) by using a Lightning Indexer to select only the top 2,048 relevant tokens per query, regardless of context length. This achieves 2-3x cost reduction for long-context inference (128K tokens) while maintaining or improving
Table of contents
Opik: Open-source LLM evaluation platformDeepSeek cracked the O(L²) attention bottleneckP.S. For those wanting to develop “Industry ML” expertise:Sort: