A Subtle Bias that Could Impact Your Decision Trees and Random Forests

This post discusses a subtle bias that can impact decision trees and random forests. The bias can be eliminated by integrating out the effect of the choice of conditioning operators. The bias is more likely to occur when a feature domain contains highly probable equidistant values and when relatively deep trees are built. Integrating out the bias involves building two forests of half the size, fitting one to the original data and the other to the mirrored data, and averaging the results.