Introduction to Approximate Solution Methods for Reinforcement Learning

Covers approximate solution methods for reinforcement learning, bridging the gap between tabular methods and real-world problems with large or continuous state spaces. Introduces function approximation as a replacement for value tables, defines the mean squared value error prediction objective, and explains how Stochastic Gradient Descent is used to minimize it. Derives gradient and semi-gradient RL algorithms including MC with function approximation and TD(0) with function approximation. Also discusses linear function approximation, feature construction techniques like polynomial and Fourier bases, and briefly touches on neural networks as non-linear approximators.