I’m neither claiming that just the architecture is reasoning, nor that the architecture would work for any task. I’m also not saying GPT is a general intelligence. I agree that GPT-3 and iGPT are separate things. However, what happens with one can be evidence for what is going on inside the other, given that they have the same architecture.
What I’m thinking is this: The path to AGI may involve “roadblocks,” i.e. things that won’t be overcome easily, i.e. things that won’t be solved simply by tweaking and recombining our existing architectures and giving them orders of magnitude more compute, data, etc. Various proposals have been made for possible roadblocks, in the form of claims about what current methods cannot do: Current methods can’t do long-term planning, current methods can’t do hidden-information games, current methods can’t do reasoning, current methods can’t do common sense, etc.
Occasionally something which is hypothesized to be a roadblock turns out not to be. E.g. it turns out AlphaStar, OpenAI Five, etc. work fine with hidden information games, and afaik this didn’t involve any revolutionary new insights but just some tweaking and recombining of existing ideas along with loads more compute.
My claim is that the GPTs are evidence against reasoning and common sense understanding being roadblocks. There may be other roadblocks. And probably GPT isn’t “reasoning” nearly as well or as comprehensively and generally as we humans do. Similarly, it’s common sense isn’t as good as mine. But it has a common sense, and it’s improving as we make bigger and bigger GPTs.
One thing I should say as a caveat is that I don’t have a clear idea of what people mean when they say reasoning is a roadblock. I think reasoning is a fuzzy and confusing concept. Perhaps I am wrong to say this is evidence against reasoning being a roadblock, because I’m misunderstanding what people mean by reasoning. I’d love to hear someone explain carefully what reasoning is and why it’s likely a roadblock.
Stuart Russell gave his list of roadblocks, which is relevant as he (might) have just made a claim that was falsified by GPT3, in that same interview -
The first thing is that the Go board is fully observable. You can see the entire state of the world that matters. And of course in the real world there’s lots of stuff you don’t see and don’t know. Some of it you can infer by accumulating information over time, what we call state estimation, but that turns out to be quite a difficult problem. Another thing is that we know all the rules of Go, and of course in the real world, you don’t know all the rules, you have to learn a lot as you go along. Another thing about the Go board is that despite the fact that we think of it as really complicated, it’s incredibly simple compared to the real world. At any given time on the Go board there’s a couple of hundred legal moves, and the game lasts for a couple hundred moves.
And if you said, well, what are the analogous primitive actions in the real world for a human being? Well, we have 600 muscles and we can actuate them maybe about 10 times per second each. Your brain probably isn’t able to do that, but physically that’s what could be your action space. And so you actually have then a far greater action space. And you’re also talking about… We often make plans that last for many years, which is literally trillions of primitive actions in terms of muscle actuations. Now we don’t plan those all out in detail, but we function on those kinds of timescales. Those are some of the ways that Go and the real world differ. And what we do in AI is we don’t say, okay, I’ve done Go, now I’m going to work on suicide Go, and now I’m going to work on chess with three queens.
What we try to do is extract the general lessons. Okay, we now understand fairly well how to handle that whole class of problems. Can we relax the assumptions, these basic qualitative assumptions about the nature of the problem? And if you relax all the ones that I listed, and probably a couple more that I’ve got
So dealing with partial observability, discovering new action sets, managing mental activity (?) and some others. This seems close to the list in an older post I wrote:
Stuart Russell’s List
human-like language comprehension
cumulative learning
discovering new action sets
managing its own mental activity
For reference, I’ve included two capabilities we already have that I imagine being on a similar list in 1960
If AlphaStar is evidence that partial observability isn’t going to be a problem, is GPT3 similarly evidence that language comprehension isn’t going to be a problem, since GPT3 can do things like simple arithmetic? That leaves cumulative learning, discovering action sets and managing mental activity on Stuart’s list.
Thanks, I agree AlphaStar doesn’t seem to have it. What do you think about GPT’s arithmetic and anagram stuff? Also, you say that AIdungeon uses GPT-3, but their “About” page still says they use GPT-2. Anyhow, I now think I was too confident in my original claim, and am further revising downwards.
AI Dungeon definitely uses GPT-3. Look at their video+blurb on the Beta page, note the updates page mentions “Double the Memory!: AI Dungeon has double the memory! If you didn’t hear, we recently upgraded our AI. With that upgrade the AI can now remember twice as much!” (there is no GPT-2 with a context window of 2048). I’ve also discussed this with Walton. I don’t know know why people find it so hard to believe that maybe a tiny startup doesn’t update every last piece of documentation instantaneously.
I’m neither claiming that just the architecture is reasoning, nor that the architecture would work for any task. I’m also not saying GPT is a general intelligence. I agree that GPT-3 and iGPT are separate things. However, what happens with one can be evidence for what is going on inside the other, given that they have the same architecture.
What I’m thinking is this: The path to AGI may involve “roadblocks,” i.e. things that won’t be overcome easily, i.e. things that won’t be solved simply by tweaking and recombining our existing architectures and giving them orders of magnitude more compute, data, etc. Various proposals have been made for possible roadblocks, in the form of claims about what current methods cannot do: Current methods can’t do long-term planning, current methods can’t do hidden-information games, current methods can’t do reasoning, current methods can’t do common sense, etc.
Occasionally something which is hypothesized to be a roadblock turns out not to be. E.g. it turns out AlphaStar, OpenAI Five, etc. work fine with hidden information games, and afaik this didn’t involve any revolutionary new insights but just some tweaking and recombining of existing ideas along with loads more compute.
My claim is that the GPTs are evidence against reasoning and common sense understanding being roadblocks. There may be other roadblocks. And probably GPT isn’t “reasoning” nearly as well or as comprehensively and generally as we humans do. Similarly, it’s common sense isn’t as good as mine. But it has a common sense, and it’s improving as we make bigger and bigger GPTs.
One thing I should say as a caveat is that I don’t have a clear idea of what people mean when they say reasoning is a roadblock. I think reasoning is a fuzzy and confusing concept. Perhaps I am wrong to say this is evidence against reasoning being a roadblock, because I’m misunderstanding what people mean by reasoning. I’d love to hear someone explain carefully what reasoning is and why it’s likely a roadblock.
Stuart Russell gave his list of roadblocks, which is relevant as he (might) have just made a claim that was falsified by GPT3, in that same interview -
So dealing with partial observability, discovering new action sets, managing mental activity (?) and some others. This seems close to the list in an older post I wrote:
If AlphaStar is evidence that partial observability isn’t going to be a problem, is GPT3 similarly evidence that language comprehension isn’t going to be a problem, since GPT3 can do things like simple arithmetic? That leaves cumulative learning, discovering action sets and managing mental activity on Stuart’s list.
I don’t know whether reasoning is a roadblock or not, but I discuss some ways in which GPT doesn’t have it in this comment.
Thanks, I agree AlphaStar doesn’t seem to have it. What do you think about GPT’s arithmetic and anagram stuff? Also, you say that AIdungeon uses GPT-3, but their “About” page still says they use GPT-2. Anyhow, I now think I was too confident in my original claim, and am further revising downwards.
AI Dungeon definitely uses GPT-3. Look at their video+blurb on the Beta page, note the updates page mentions “Double the Memory!: AI Dungeon has double the memory! If you didn’t hear, we recently upgraded our AI. With that upgrade the AI can now remember twice as much!” (there is no GPT-2 with a context window of 2048). I’ve also discussed this with Walton. I don’t know know why people find it so hard to believe that maybe a tiny startup doesn’t update every last piece of documentation instantaneously.
OK, thanks. I don’t find that hard to believe at all.
Ah I see, that makes more sense, thanks!