Developing the Right Intuition for Adaboost From Scratch

Have you ever wondered while going through Kaggle notebooks how the top competitors come with brilliant models with high accuracy? The answer simply to that is the use of ensemble methods in their models. Adaboost is an iterative ensemble technique in which the whole data is fed into several base models(weak classifiers) and the final prediction is made by calculating the weighted sum of these base models. The most interesting feature about this algorithm is that each base model is created in such a way that each of the base models learns from the mistakes made by the previous ones and as a result of this, it iteratively reduces the training error gradually and improves the classification performance tremendously. Adaboost over the years has helped in developing face detection technology, hand detection technology, and human detection. I have tried in this blog to help you develop an intuition about the classifier and how to make it from scratch.

To understand the working of Adaboost let us first understand what is meant by ensemble techniques. In this method first, we divide the data given to us into child datasets and then the datasets are fed into various base models (like SVM, logistic regression) and then the result is predicted by aggregating the results predicted by all of these bases.

Ensemble methods were introduced to convert the Low Biased High Variance model to the Low Biased Low Variance model.

Low Biased High Variance model is one that performs well in the training data (that’s why low biased) but as soon as some data is provided outside of training data the model becomes faulty(that’s why high variance). It has been found out that…

Related posts:

The 21st century is considered a century of technological breakthroughs that occur daily. Digital technology is increasingly capturing the life of modern users, forcing them to reconsider their views…

When we put something on a pedestal, we create a kind of resistance that keeps us from getting it. It keeps us further away, and in the process keeps us away because we want it even more. If this is…

No es nada personal pero solo es el comienzo. “Quiero hacer una prueba” is published by Maurino Carbajal.