ensemble learning

idea: train multiple classifier and then combine them to improve performance.

aggregate their decisions via voting procedure.

Think of boosting, decision tree.

bagging

using non-overlapping training subset creates truly independent/diverse classifiers

bagging is essentially bootstrap aggregating where we do random sampling with replacement.

bagging but with random subspace methods ¹

NOTE

can overfit easily with deeper tree.

a greedier approach for reducing bias where we “pick base classifiers incrementally”.

we will train “weaker learner” and thus it can combined to become “stronger learner”.

The idea of training each classifier using a random subset of the feature sets. Also known as feature bagging ↩