We only use boosting if our set of low-complexity hypotheses does not contain the solution we need. And instead of switching to a larger set of still-low-complexity hypotheses, we do something much cheaper, a second best thing: we try to find a good hypothesis in the convex hull of the original hypothesis space.
In short, boosting “outperforms Occam” only in man-hours saved: boosting requires less thinking and less work than properly applying Occam’s razor. That really is a good thing, of course.
We only use boosting if our set of low-complexity hypotheses does not contain the solution we need. And instead of switching to a larger set of still-low-complexity hypotheses, we do something much cheaper, a second best thing: we try to find a good hypothesis in the convex hull of the original hypothesis space.
In short, boosting “outperforms Occam” only in man-hours saved: boosting requires less thinking and less work than properly applying Occam’s razor. That really is a good thing, of course.