Billion-Parameter Theories

A philosophical essay arguing that complex systems like climate, poverty, and disease were never intractable — they simply required theories too large for human minds to hold. The author contends that large neural networks are legitimate scientific theories of complex systems, just expressed in billions of parameters rather than elegant equations. The piece distinguishes between complicated systems (reducible to components) and complex ones (emergent, feedback-driven), argues that transformer architectures represent a compact meta-theory with broad reach, and suggests mechanistic interpretability may become the new science of complexity — extracting compressible truths by studying trained models as specimens rather than deriving equations from first principles.