I like the post and generally agree. Here’s a random thought on the OOD generalization. I feel that often we talk about how being good at 2 or 3 different things allow for new exploration. If you believe in books such as Range, then we’re a lot more creative when combining ideas from multiple different fields. I rather think of multiple “hulls” (I’m guessing this isn’t technically correct since I’m a noob at convex optimisation.) and how to apply them together to find new truths.
Well, those three sets of points ultimately still define only one hull. But I get your intuition—there are areas inside that hull that are high density, and areas that are much lower density (but in which it might be easier to extrapolate due to being surrounded by known areas). I feel like also our inability to visualize these things in their true dimensionality is probably really limiting. The real structures must be mind-boggling and probably looking more like some kind of fractal filament networks.
I like the post and generally agree. Here’s a random thought on the OOD generalization. I feel that often we talk about how being good at 2 or 3 different things allow for new exploration. If you believe in books such as Range, then we’re a lot more creative when combining ideas from multiple different fields. I rather think of multiple “hulls” (I’m guessing this isn’t technically correct since I’m a noob at convex optimisation.) and how to apply them together to find new truths.
Well, those three sets of points ultimately still define only one hull. But I get your intuition—there are areas inside that hull that are high density, and areas that are much lower density (but in which it might be easier to extrapolate due to being surrounded by known areas). I feel like also our inability to visualize these things in their true dimensionality is probably really limiting. The real structures must be mind-boggling and probably looking more like some kind of fractal filament networks.