The limitations of direct optimisation in rich environments seem complexity theoretic, so better algorithms won’t fix them
This seems questionable. Humans are pretty good at “direct optimization” when they want to be (a point mentioned in the original post).
And it seems straightforward to construct artificial systems that behave even more like “direct optimizers” than humans, even if some or all of the component pieces of those systems are made out of function-approximators. Mu Zero is a good example; I sketched what a “real world” version might look like here.
To me, “amortized optimization” seems like just one tool in the toolbox of actual optimization, which is about choosing actions that steer towards outcomes.
Humans aren’t pure direct optimisers, though I think there’s a point about using abstractions to simplify a problem or translate it to a simple domain that’s more amenable to direct optimisation.
This seems questionable. Humans are pretty good at “direct optimization” when they want to be (a point mentioned in the original post).
And it seems straightforward to construct artificial systems that behave even more like “direct optimizers” than humans, even if some or all of the component pieces of those systems are made out of function-approximators. Mu Zero is a good example; I sketched what a “real world” version might look like here.
To me, “amortized optimization” seems like just one tool in the toolbox of actual optimization, which is about choosing actions that steer towards outcomes.
Humans aren’t pure direct optimisers, though I think there’s a point about using abstractions to simplify a problem or translate it to a simple domain that’s more amenable to direct optimisation.