I think the response I’m most sympathetic to is something like “yes, currently to get human-level results you need to bake in a lot of knowledge, but as time goes on we will need less and less of this”. For example, instead of having a fixed form for the objective, you could have a program sketch in a probabilistic programming language. Instead of hardcoded state observations, you use the features from a learned model. In order to cross the superexponential barrier, you also sprinkle neural networks around; instead of using a small grammar of possible programs, you use the distribution induced by something like Codex, when doing probabilistic inference, you train a neural network to predict the output of the inference, etc. There are lots of details to be filled in here, but that is true in any path to AGI.
(Though I still generally expect AGI where the main ingredient is “scaled up neural networks”.)
Yeah, I should perhaps have emphasized this more.
I think the response I’m most sympathetic to is something like “yes, currently to get human-level results you need to bake in a lot of knowledge, but as time goes on we will need less and less of this”. For example, instead of having a fixed form for the objective, you could have a program sketch in a probabilistic programming language. Instead of hardcoded state observations, you use the features from a learned model. In order to cross the superexponential barrier, you also sprinkle neural networks around; instead of using a small grammar of possible programs, you use the distribution induced by something like Codex, when doing probabilistic inference, you train a neural network to predict the output of the inference, etc. There are lots of details to be filled in here, but that is true in any path to AGI.
(Though I still generally expect AGI where the main ingredient is “scaled up neural networks”.)