I agree with the premise/ontology but disagree with the conclusion. It’s pretty clear the generator of disagreement is
I believe GPT has demonstrated that near-human-level AI is a possible steady state for multiple years
I don’t think it’s been a steady state nor that it will proceed as one; rather I think GPT not only marks but is causally responsible for the beginning of accelerating takeoff and is already a gateway to a variety of superhuman capabilities (or “products”).
[a few minutes later] Actually, I realized I do agree, in the sense of relative updates. I’ve gotten too used to framing my own views in contrast to others’ rather than in relation to my past self. GPT surprised me by demonstrating that an AI can be as intelligent as it is and deployed over the internet, allowed to execute code, etc, without presenting an immediate existential threat. Before I would have guessed that by the time an AI exists which uses natural language as fluently as GPT-3 we’d have months to minutes left, lol. The update was toward slower takeoff (but shorter timelines because I didn’t think any of this would happen so soon).
But from this post, which feels a bit fragmented, I’m not clear on how the conclusions about timelines and x-risk follow from the premise of platform/product distinction, and I’d like to better understand the thread as you see it.
Thanks for feedback, I am new to writing in this style and may have erred too much towards deleting sentences while editing. But, if you never cut too much you’re always too verbose, as they say. I in particular appreciate that, when talking about how I am updating, I should make clear where I am updating from.
For instance, regarding human level intelligence, I was also describing relative to “me a year/month ago”. I relistened to the Sam Harris/Yudkowsky podcast yesterday, and they detour for a solid 10 minutes about how “human level” intelligence is a straw target. I think their arguments were persuasive, and that I would have endorsed them a year ago, but that they don’t really apply to GPT. I had pretty much concluded that the difference between a 150 IQ AI and a 350 IQ AI would be a matter of scale. GPT as a simulator/platform seems to me like an existence proof for a not-artificially-handicapped human level AI attractor state. Since I had previous thought the entire idea was a distraction, this is an update towards human level AI.
The impact on AI timelines mostly follows from diversion of investment. I will think on if I have anything additional to add on that front.
I understand your reasoning much better now, thanks!
“GPT as a simulator/platform seems to me like an existence proof for a not-artificially-handicapped human level AI attractor state” is a great way to put it and a very important observation.
I think the attractor state is more nuanced than “human-level”. GPT is incentivized to learn to model “everyone everywhere all at once” if you will, a superhuman task—and while the default runtime behavior is human-level simulacra, I expect it to be possible to elicit superhuman performance by conditioning the model in certain ways or a relatively small amount of fine tuning/RL. Also, being simulated confers many advantages for intelligence (instances can be copied/forked, are much more programmable than humans, potentially run much faster, etc). So I generally think of the attractor state as being superhuman in some important dimensions, enough to be a serious foom concern.
Broadly, though, I agree with the framing—even if it’s somewhat superhuman, it’s extremely close to human-level and human-shaped intelligence compared to what’s possible in all of mindspace, and there is an additional unsolved technical challenge to escalate from human-level/slightly superhuman to significantly beyond that. You’re totally right that it removes the arbitrariness of “human-level” as a target/regime.
I’d love to see an entire post about this point, if you’re so inclined. Otherwise I might get around to writing something about it in a few months, lol.
I agree with the premise/ontology but disagree with the conclusion. It’s pretty clear the generator of disagreement is
I don’t think it’s been a steady state nor that it will proceed as one; rather I think GPT not only marks but is causally responsible for the beginning of accelerating takeoff and is already a gateway to a variety of superhuman capabilities (or “products”).
[a few minutes later] Actually, I realized I do agree, in the sense of relative updates. I’ve gotten too used to framing my own views in contrast to others’ rather than in relation to my past self. GPT surprised me by demonstrating that an AI can be as intelligent as it is and deployed over the internet, allowed to execute code, etc, without presenting an immediate existential threat. Before I would have guessed that by the time an AI exists which uses natural language as fluently as GPT-3 we’d have months to minutes left, lol. The update was toward slower takeoff (but shorter timelines because I didn’t think any of this would happen so soon).
But from this post, which feels a bit fragmented, I’m not clear on how the conclusions about timelines and x-risk follow from the premise of platform/product distinction, and I’d like to better understand the thread as you see it.
Thanks for feedback, I am new to writing in this style and may have erred too much towards deleting sentences while editing. But, if you never cut too much you’re always too verbose, as they say. I in particular appreciate that, when talking about how I am updating, I should make clear where I am updating from.
For instance, regarding human level intelligence, I was also describing relative to “me a year/month ago”. I relistened to the Sam Harris/Yudkowsky podcast yesterday, and they detour for a solid 10 minutes about how “human level” intelligence is a straw target. I think their arguments were persuasive, and that I would have endorsed them a year ago, but that they don’t really apply to GPT. I had pretty much concluded that the difference between a 150 IQ AI and a 350 IQ AI would be a matter of scale. GPT as a simulator/platform seems to me like an existence proof for a not-artificially-handicapped human level AI attractor state. Since I had previous thought the entire idea was a distraction, this is an update towards human level AI.
The impact on AI timelines mostly follows from diversion of investment. I will think on if I have anything additional to add on that front.
I understand your reasoning much better now, thanks!
“GPT as a simulator/platform seems to me like an existence proof for a not-artificially-handicapped human level AI attractor state” is a great way to put it and a very important observation.
I think the attractor state is more nuanced than “human-level”. GPT is incentivized to learn to model “everyone everywhere all at once” if you will, a superhuman task—and while the default runtime behavior is human-level simulacra, I expect it to be possible to elicit superhuman performance by conditioning the model in certain ways or a relatively small amount of fine tuning/RL. Also, being simulated confers many advantages for intelligence (instances can be copied/forked, are much more programmable than humans, potentially run much faster, etc). So I generally think of the attractor state as being superhuman in some important dimensions, enough to be a serious foom concern.
Broadly, though, I agree with the framing—even if it’s somewhat superhuman, it’s extremely close to human-level and human-shaped intelligence compared to what’s possible in all of mindspace, and there is an additional unsolved technical challenge to escalate from human-level/slightly superhuman to significantly beyond that. You’re totally right that it removes the arbitrariness of “human-level” as a target/regime.
I’d love to see an entire post about this point, if you’re so inclined. Otherwise I might get around to writing something about it in a few months, lol.