databricks

Training high-quality machine learning models involves careful data preparation, which can be time-consuming for large datasets. The Photon Engine, now available in Databricks Machine Learning Runtime, significantly speeds up Spark SQL and Spark DataFrame workloads, achieving speed improvements of 2x-4x. The Photon Engine enhances ETL processes and feature engineering, especially for time series data, using a new point-in-time join implementation. Users can enable Photon in Databricks ML Runtime 15.2 and above for better query performance.

Accelerate Feature Engineering With Photon

How does Photon help machine learning workloads?

Select Photon in Databricks Machine Learning Runtime cluster