The RL Irony in LLMs (And its insane new Meta)

Reinforcement learning (RL) has become crucial for improving LLM capabilities like coding and reasoning, despite experts claiming it won't lead to AGI. RL provides sparse signals (one bit per episode) compared to dense next-token prediction, making it computationally efficient but less generalizable. Recent research shows RL updates only 5% of model weights, making it compatible with LoRA (Low-Rank Adaptation). When properly configured with LoRA on all layers, 10x higher learning rates, and moderate batch sizes, RL training matches full fine-tuning performance while using only 2/3 of the compute. This combination enables efficient experimentation and personalized AI agents at scale, potentially making specialized capabilities widely accessible without achieving true AGI.