Ensemble Learning has garnered a lot of attention in recent years. Instead of searching for the elusive “perfect” model, data scientists have gravitated towards building a number of diverse “acceptable” models that collectively outperform the best of these singular models. However, creating robust ensembles often borders on wizardry as we need to optimize dozens of tuning parameters for each model over high dimensional search spaces. We’ll discuss popular Ensembling techniques that are widely popular and several tricks that practitioners employ in selecting parameters to improve generalizability over a single model.
