I submitted a tiny patch to flash attention. The necessary typing for the change takes less ten seconds, but the overall change took more than ten hours So where does the time go? It started when coworker had a bug where cudnn attention would crash randomly. We looked at his unreleased changes and concluded that…

Programming Digest is a curated collection of articles, tutorials, and resources for software developers covering a wide range of programming languages, frameworks, and technologies. With a focus on practical tips, code snippets, and industry insights, Programming Digest provides a resource for developers looking to stay updated with the latest trends and advancements in software development.

Programming Digest

A developer chronicles how a 10-second code fix to flash-attention took over 10 hours spread across multiple days. The narrative walks through 14 steps including fighting sandboxed test environments, CUDA version conflicts, OOM build failures, wrong GPU compute capability, compiler segfaults, incremental build failures, and navigating a codebase refactor that hid the bug. The post concludes that most programmer time is lost to: punching through abstraction layers, fighting build systems, managing dependencies, and running wrong code versions — and that LLM tools still don't meaningfully help with these non-coding tasks.

How Programmers Spend Their Time