Maybe I misunderstand your use of robust, but this still seems to me to be breadth. If an optima is broader, samples are more likely to fall within it. I took broad to mean “has a lot of (hyper)volume in the optimization space”, and robust to mean “stable over time/perturbation”. I still contend that those optimization processes are unaware of time, or any environmental variation, and can only select for it in so far as it is expressed as breadth.
The example I have in my head is that if you had an environment, and committed to changing some aspect of it after some period of time, evolution or SGD would optimize the same as if you had committed to a different change. Which change you do would affect the robustness of the environmental optima, but the state of the environment alone determines their breadth. The processes cannot optimize based on your committed change before it happens, so they cannot optimize for robustness.
Given what you said about random samples, I think you might be working under definitions along the lines of “robust optima are ones that work in a range of environments, so you can be put in a variety of random circumstances, and still have them work” and (at this point I struggled a bit to figure out what a “broad” optima would be that’s different, and this is what I came up with?) “broad optima are those that you can do approximately and still get a significant chunk of the benefit.” I feel like these can still be unified into one thing, because I think approximate strategies in fixed environments are similar to fixed strategies in approximate environments? Like moving a little to the left is similar to the environment being a little to the right?
Maybe I misunderstand your use of robust, but this still seems to me to be breadth. If an optima is broader, samples are more likely to fall within it. I took broad to mean “has a lot of (hyper)volume in the optimization space”, and robust to mean “stable over time/perturbation”. I still contend that those optimization processes are unaware of time, or any environmental variation, and can only select for it in so far as it is expressed as breadth.
The example I have in my head is that if you had an environment, and committed to changing some aspect of it after some period of time, evolution or SGD would optimize the same as if you had committed to a different change. Which change you do would affect the robustness of the environmental optima, but the state of the environment alone determines their breadth. The processes cannot optimize based on your committed change before it happens, so they cannot optimize for robustness.
Given what you said about random samples, I think you might be working under definitions along the lines of “robust optima are ones that work in a range of environments, so you can be put in a variety of random circumstances, and still have them work” and (at this point I struggled a bit to figure out what a “broad” optima would be that’s different, and this is what I came up with?) “broad optima are those that you can do approximately and still get a significant chunk of the benefit.” I feel like these can still be unified into one thing, because I think approximate strategies in fixed environments are similar to fixed strategies in approximate environments? Like moving a little to the left is similar to the environment being a little to the right?