Bagging in overcoming variance of a classifier, clustering algorithm or regressor

Bootstrap aggregating

Bootstrap aggregating, also called bagging, is a machine learning ensemble meta-algorithm designed to improve the stability and accuracy of machine learning algorithms used in statistical classification and regression. It also reduces variance and helps to avoid overfitting. Although it is usually applied to decision tree methods, it can be used with any type of method. Bagging is a special case of the model averaging approach. 

Bagging: Sampling with replacement

Build classifier on each bootstrap sample
Each sample has probability (1 - (1 - 1/n) ^ n) of being selected in training set.

Sampling data for bagging



Probability that a sample will be selected in m rounds of bagging



Probability that ensemble will make an error with majority voting decision making technique

Tags: Technology,Machine Learning,Classification,Clustering,Regression,