This is related to making a strong learner (really accurate) out of weak learners (barely better than majority).
It is actually somewhat non-obvious this should even be possible.
The famous example here is boosting, and in particular “AdaBoost.” The reason boosting et al. work well is actually kind of interesting and I think still not entirely understood.
I didn’t really get Vaniver’s explanation below, there are margin methods that draw the line in a sensible way that have nothing to do with weak learners at all.
This is related to making a strong learner (really accurate) out of weak learners (barely better than majority). It is actually somewhat non-obvious this should even be possible.
The famous example here is boosting, and in particular “AdaBoost.” The reason boosting et al. work well is actually kind of interesting and I think still not entirely understood.
I didn’t really get Vaniver’s explanation below, there are margin methods that draw the line in a sensible way that have nothing to do with weak learners at all.