A Neighborhood of Infinity: A Simple Switch Makes Code Differentiable

A deep-dive into how switching algebraic types in Haskell can make code automatically differentiable 'for free', without rewriting algorithms. Using the vector space monad (V s a) as a drop-in replacement for lists, a simple dictionary insert operation becomes a differentiable tensor product update. The post connects this technique to the DeltaNet update rule from recent ML research on parallelizing linear transformers with the delta rule, and draws parallels to adjoint-mode automatic differentiation and the REDUCE algorithm for stochastic processes.