@adlrocha - What if AI doesn’t need more RAM but better math?

Google released TurboQuant, a two-stage KV cache compression algorithm that achieves a 6x reduction in memory usage with no measurable accuracy loss. Stage 1 (PolarQuant) converts vectors from Cartesian to polar coordinates, exploiting the predictable angular distribution in transformer key spaces to compress without calibration data. Stage 2 (QJL) applies a Johnson-Lindenstrauss transform to correct quantization error at zero memory overhead. The result is 3.5 bits per channel with quality neutrality across major models, and up to 8x performance improvement on H100 GPUs. Unlike other quantization methods, TurboQuant is data-oblivious and requires no fine-tuning. Beyond LLMs, it shows promise for vector databases, RAG pipelines, recommendation engines, and on-device inference. The announcement caused memory stock prices (Micron, SanDisk) to drop, raising questions about whether AI's memory demand will grow as linearly as previously assumed.