Cloudflare discovered a race condition in Go's arm64 compiler that caused sporadic crashes during stack unwinding. The bug occurred when async preemption interrupted a split stack pointer adjustment in function epilogues, leaving the stack in an invalid state. Through systematic debugging involving coredumps, disassembly analysis, and reproducer creation, they traced crashes to a one-instruction race where the runtime preempted between two ADD opcodes adjusting RSP. The fix ensures stack pointer modifications happen atomically by building offsets in temporary registers first.

16m read timeFrom blog.cloudflare.com
Post cover image
Table of contents
Investigating a strange panicBreakthroughBuilding a minimal reproducerA single-instruction race condition window
1 Comment

Sort: