Online Learning or Don’t Look Ahead

Andrey Gulin
Russia, Yandex
Boosting builds complex models by combining many simple ones. But what if we want to combine many complex models to build an even better model? We will consider this task on the problem of constructing predictors for data with high cardinality category features. It turns out that a direct approach does not work out but a few modifications allow us to reach the target. The proposed algorithm can successfully combine arbitrary complex models in a boosting-like fashion. The same algorithm achieves competitive results when used for combining simple models like trees compared to generic tree boosting.