Accelerate Feature Engineering With Photon

Training high-quality machine learning models involves careful data preparation, which can be time-consuming for large datasets. The Photon Engine, now available in Databricks Machine Learning Runtime, significantly speeds up Spark SQL and Spark DataFrame workloads, achieving speed improvements of 2x-4x. The Photon Engine enhances ETL processes and feature engineering, especially for time series data, using a new point-in-time join implementation. Users can enable Photon in Databricks ML Runtime 15.2 and above for better query performance.