I agree; a relatively slow “foom” is likely; moreover, the human team(s) doing that will know that this is exactly what they are doing, a “slowish” foom (for 2 OOM (+/-1) per 6-12 months; still way faster than our current rate of progress).
Whether this process can unexpectedly run away from them and explode really fast instead at some point would depend on whether completely unexpected radical algorithmic discoveries will be made in the process (that’s one thing the whole ecosystem of humans+AIs in an organization like that should watch for; they need to have genuine consensus among involved humans and involved AIs to collectively ponder such things before allowing them to accelerate beyond a “slowish” foom to a much faster one; but it’s not certain if the discoveries enabling the really fast one will be made, it’s just a possibility).
Yep, agreed. Stronger-than-expected jump unlikely but possible and should be guarded against.
As for the 2 OOM speed.… I agree, it’s substantially faster than what we’ve been experiencing so far. Think of GPT4 getting 100x stronger/smarter over the course of a year. That’s plenty enough to be scary I think.
I agree; a relatively slow “foom” is likely; moreover, the human team(s) doing that will know that this is exactly what they are doing, a “slowish” foom (for 2 OOM (+/-1) per 6-12 months; still way faster than our current rate of progress).
Whether this process can unexpectedly run away from them and explode really fast instead at some point would depend on whether completely unexpected radical algorithmic discoveries will be made in the process (that’s one thing the whole ecosystem of humans+AIs in an organization like that should watch for; they need to have genuine consensus among involved humans and involved AIs to collectively ponder such things before allowing them to accelerate beyond a “slowish” foom to a much faster one; but it’s not certain if the discoveries enabling the really fast one will be made, it’s just a possibility).
Yep, agreed. Stronger-than-expected jump unlikely but possible and should be guarded against. As for the 2 OOM speed.… I agree, it’s substantially faster than what we’ve been experiencing so far. Think of GPT4 getting 100x stronger/smarter over the course of a year. That’s plenty enough to be scary I think.