Online Offline Feature Skew in Machine Learning

The Math of Model Generalization assumes that the probability distribution of the features seen at the training time is identical to the distribution of features at the inference time. This is scary because training performance is often the only reliable compass that points to the true north of production performance. We need to identify and eliminate all the causes of training inference skew.