bagging

Ensembles dominate tabular ML, and this question tests whether you grasp that they attack different parts of the error. The signal is bagging-reduces-variance vs boosting-reduces-bias and why GBMs win on tabular data. Here is the answer.

Ensembles win Kaggle and quietly drive most production tabular models. What matters is explaining why combining models beats one (error decorrelation), and knowing exactly how the four techniques differ in what they fix.

Anyone can say 'ensemble of trees'. The real answer concerns decorrelating those trees and why that, not averaging alone, is what drops variance. Here is what makes the forest more than the sum of its trees.