A comprehensive guide to cleaning time series data in Python, covering the full pipeline from raw data to model-ready datasets. Topics include auditing the time index, reindexing to canonical frequency, handling missing values with forward fill, time interpolation, and seasonal decomposition, detecting and treating outliers using rolling Z-score, IQR, and Isolation Forest, removing duplicate timestamps, resampling across frequencies, smoothing noise with EWMA and Savitzky-Golay filters, and validating the cleaned data with schema checks. All techniques are demonstrated with sensor data examples and runnable code.
Table of contents
PrerequisitesTable of ContentsHow to Audit Your Time Series Before Cleaning ItHow to Reindex to a Canonical FrequencyHow to Handle Missing ValuesHow to Detect and Handle OutliersHow to Remove DuplicatesFrequency Alignment and ResamplingSmoothing NoiseSchema and Sanity ValidationThe Complete Cleaning ChecklistWrapping UpSort: