Linear regression can be understood geometrically as a projection problem in column space rather than a calculus optimization in feature space. Instead of treating data points as rows on a scatter plot, each observation becomes an axis, turning feature columns into vectors. The goal of regression becomes finding the closest point on the plane spanned by the feature vectors (Size and Base/intercept) to the target Price vector. This closest point is found via orthogonal projection, producing the Prediction Vector and a Residual Vector perpendicular to the feature plane. This geometric view scales elegantly to any number of features and naturally derives the Normal Equation without calculus.
Table of contents
The Usual Way: Feature SpaceA Shift in PerspectiveFrom Points to DirectionsWhy This Perspective MattersThe Geometry Behind RegressionA Simple Way to See ThisWhy We Need a Base Value?Combining DirectionsThe Space of All Possible PredictionsFrom a Point to a VectorThe Vector SubtractionLinear Regression Is a ProjectionWhat Changed Was the PerspectiveSort: