Bagging, short for bootstrap aggregating, is an ensemble learning technique primarily used to improve the stability and accuracy of machine learning algorithms, particularly decision trees. The core idea behind bagging is to reduce variance by training multiple models on different subsets of the training data and then aggregating their predictions.

The process involves creating several bootstrapped samples from the original dataset, where each sample is generated by randomly selecting observations with replacement. For each of these samples, a separate model (often a decision tree) is trained. Finally, the predictions from all the models are combined—averaging for regression tasks and voting for classification tasks—to produce a final prediction. This approach helps to mitigate the overfitting problem commonly associated with individual decision trees ²³⁴.

Bagging is particularly effective because it leverages the diversity of the models trained on different data subsets, leading to improved generalization on unseen data ¹⁵.