Models built using deep learning are a function of the learning algorithm, the architecture, and the task / environment / dataset. While a lot of effort is spent on analyzing learning algorithms and architectures, not much is spent on the environment. This post asks how important it is to design a good environment in order to build AGI.
It considers two possibilities: “easy paths”, in which many environments would incentivize AGI, and “hard paths”, in which such environments are rare. (Note that “hard paths” can be true, even if an AGI would be optimal for most environments: if AGI would be optimal, but there is no path in the loss landscape to AGI that is steeper than other paths in the loss landscape, then we probably wouldn’t find AGI in that environment.)
The main argument for “hard paths” is to look at the history of AI research, where we often trained agents on tasks that were “hallmarks of intelligence” (like chess) and then found that the resulting systems were narrowly good at the particular task, but were not generally intelligent. You might think that it can’t be too hard, since our environment led to the creation of general intelligence (us), but this is subject to anthropic bias: only worlds with general intelligence would ask whether environments incentivize general intelligence, so they will always observe that their environment is an example that incentivizes general intelligence. It can serve as a proof of existence, but not as an indicator that it is particularly likely.
Planned opinion:
I think this is an important question for AI timelines, and the plausibility of “hard paths” is one of the central reasons that my timelines are longer than others who work on deep learning-based AGI. However, <@GPT-3@>(@Language Models are Few-Shot Learners@) demonstrates quite a lot of generality, so recently I’ve started putting more weight on “actually, designing the environment won’t be too hard”, which has correspondingly shortened my timelines.
+1, I endorse this summary. I also agree that GPT-3 was an update towards the environment not mattering as much as I thought.
Your summary might be clearer if you rephrase as:
It considers two possibilities: the “easy paths hypothesis” that which many environments would incentivize AGI, and the “hard paths hypothesis” that such environments are rare.
Since “easy paths” and “hard paths” by themselves are kinda ambiguous terms—are we talking about the paths, or the hypothesis? This is probably my fault for choosing bad terminology.
Planned summary for the Alignment Newsletter:
Planned opinion:
+1, I endorse this summary. I also agree that GPT-3 was an update towards the environment not mattering as much as I thought.
Your summary might be clearer if you rephrase as:
It considers two possibilities: the “easy paths hypothesis” that which many environments would incentivize AGI, and the “hard paths hypothesis” that such environments are rare.
Since “easy paths” and “hard paths” by themselves are kinda ambiguous terms—are we talking about the paths, or the hypothesis? This is probably my fault for choosing bad terminology.
Done :)