How to Inspect Decision Trees After Training with PCA

Decision trees often create perpendicular split conditions which can lead to overfitting, particularly with diagonal decision boundaries. Running PCA before fitting a decision tree can project data into orthogonal space, potentially reducing the tree's depth and improving performance. However, PCA components are not interpretable, which can be a limitation in some cases. Proper feature engineering might be necessary for better model performance.