LaurieWired·7wYour RAM freezes 128,000 times per second (I fixed it)
DRAM refresh cycles (TRFC) cause periodic memory stalls every ~3.9 microseconds, adding up to 400-500ns of latency when a read coincides with a refresh. This deep-dive explores a novel technique called 'Tail Slayer' that reduces P99.99 tail latency by 7-15x. The approach uses hedged reads: duplicating data across independent memory channels so two threads on separate cores race to read the same data, and whichever avoids the refresh stall wins. Key challenges covered include virtual vs physical memory addressing, huge pages, reverse-engineering undocumented CPU memory controller XOR hash functions (for AMD Zen4/5, Intel Sapphire/Granite Rapids, and AWS Graviton ARM) using hardware performance counters and statistical timing analysis. Benchmarks on bare-metal cloud instances confirm dramatic tail latency improvements across DDR4/DDR5, x86, and ARM architectures. The technique is particularly relevant for high-frequency trading systems where a single DRAM stall can cost millions.
