Ok, I see. Now, regarding your disjunction earlier of (in my words)
A. (NAA (nonAGI AI) takeover) You can get strategic takeover AI without AGI
B. (AGI soon easy) Gippity+ will soon be AGI by adding a bit more ~mundane human research juice
C. (AGI soon hard) Gippity+++ will soon be AGI by adding a couple big insights
First, to clarify, I think the discourse on this thread is that I asked you about
“why talk about nonAGI AI”
and you said
“because 2) maybe we already have AGI basically (scenario B, AGI soon easy), and because 1) you could get transformative AI with current nonAGI AI (scenario A, NAA takeover)”,
and now we are discussing what NAA looks like and how timelines look in NAA takeover world.
Now I’m wondering, what are your very approximate relative probabilities of these things? E.g. is one of them 90% of the source of your confidence (I mean, 90% of your prob mass) in FOOM within 10 years? If they are roughly equal, I would raise my eyebrow and say “that seems kinda strange, unless there’s a shared factor such as you thinking that actually we basically have ~AGI in current systems; if so, could you clarify that shared factor”.
As stated, these don’t have to sum to 1. B and C are mutually exclusive but A can be true even if B or C are also true.
(I also object a bit to calling “strong fluid intelligence” “AGI.”
Part of what’s at stake is how far can you get with basically just specialized knowledge and the ability to train new specialized knowledge. It would be surprising to me, but not out of the question, that there’s almost nothing that such an AI can’t do that an AI with more fluid intelligence can do. But I only object a bit.)
Ass numbers:
A: 80%
B: 40%
C: 30%
If they are roughly equal, I would raise my eyebrow and say “that seems kinda strange, unless there’s a shared factor such as you thinking that actually we basically have ~AGI in current systems; if so, could you clarify that shared factor”.
I mean that’s kind of fair. But I in fact don’t have a lot of precise ability to distinguish between “one key idea is missing” and “only engineering schlep is missing”. Those wolds look very similar, to me, and so get similar amounts of mass.
Ok, I see. Now, regarding your disjunction earlier of (in my words)
A. (NAA (nonAGI AI) takeover) You can get strategic takeover AI without AGI
B. (AGI soon easy) Gippity+ will soon be AGI by adding a bit more ~mundane human research juice
C. (AGI soon hard) Gippity+++ will soon be AGI by adding a couple big insights
First, to clarify, I think the discourse on this thread is that I asked you about
and you said
and now we are discussing what NAA looks like and how timelines look in NAA takeover world.
Now I’m wondering, what are your very approximate relative probabilities of these things? E.g. is one of them 90% of the source of your confidence (I mean, 90% of your prob mass) in FOOM within 10 years? If they are roughly equal, I would raise my eyebrow and say “that seems kinda strange, unless there’s a shared factor such as you thinking that actually we basically have ~AGI in current systems; if so, could you clarify that shared factor”.
As stated, these don’t have to sum to 1. B and C are mutually exclusive but A can be true even if B or C are also true.
(I also object a bit to calling “strong fluid intelligence” “AGI.”
Part of what’s at stake is how far can you get with basically just specialized knowledge and the ability to train new specialized knowledge. It would be surprising to me, but not out of the question, that there’s almost nothing that such an AI can’t do that an AI with more fluid intelligence can do. But I only object a bit.)
Ass numbers:
A: 80%
B: 40%
C: 30%
I mean that’s kind of fair. But I in fact don’t have a lot of precise ability to distinguish between “one key idea is missing” and “only engineering schlep is missing”. Those wolds look very similar, to me, and so get similar amounts of mass.