My Eliezer-model thinks that “there will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles” is far less than 30% likely, because it’s so conjunctive:
It requires that there ever be a one-year interval in which the world output doubles.
It requires that there be a preceding four-year interval in which world output doubles.
So, it requires that the facts of CS be such that we can realistically get AI tech that capable before the world ends...
… and separately, that this capability not accelerate us to superintelligent AI in under four years...
… and separately, that ASI timelines be inherently long enough that we don’t incidentally get ASI within four years anyway.
Separately, it requires that individual humans make the basic-AI-research decisions to develop that tech before we achieve ASI. (Which may involve exercising technological foresight, making risky bets, etc.)
Separately, it requires that individual humans leverage that tech to intelligently try to realize a wide variety of large economic gains, before we achieve ASI. (Which may involve exercising technological, business, and social foresight, making risky bets, etc.)
Separately, it requires that the regulatory environment be favorable.
(Possibly other assumptions are required here too, like ‘the first groups that get this pre-AGI tech even care about transforming the world economy, vs. preferring to focus on more basic research, or alignment / preparation-for-AGI, etc.’)
You could try to get multiple of those properties at once by assuming specific things about the world’s overall adequacy and/or about the space of all reachable intelligent systems; but from Eliezer’s perspective these views fall somewhere on the spectrum between ‘unsupported speculation’ and ‘flatly contradicted by our observations so far’, and there are many ways to try to tweak civilization to be more adequate and/or the background CS facts to be more continuous, and still not hit the narrow target “a complete 4 year interval in which world output doubles” (before AGI destroys the world or a pivotal act occurs).
(I’m probably getting a bunch of details about Eliezer’s actual model wrong above, but my prediction is that his answer will at least roughly look like this.)
My Eliezer-model thinks that “there will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles” is far less than 30% likely, because it’s so conjunctive:
It requires that there ever be a one-year interval in which the world output doubles.
It requires that there be a preceding four-year interval in which world output doubles.
So, it requires that the facts of CS be such that we can realistically get AI tech that capable before the world ends...
… and separately, that this capability not accelerate us to superintelligent AI in under four years...
… and separately, that ASI timelines be inherently long enough that we don’t incidentally get ASI within four years anyway.
Separately, it requires that individual humans make the basic-AI-research decisions to develop that tech before we achieve ASI. (Which may involve exercising technological foresight, making risky bets, etc.)
Separately, it requires that individual humans leverage that tech to intelligently try to realize a wide variety of large economic gains, before we achieve ASI. (Which may involve exercising technological, business, and social foresight, making risky bets, etc.)
Separately, it requires that the regulatory environment be favorable.
(Possibly other assumptions are required here too, like ‘the first groups that get this pre-AGI tech even care about transforming the world economy, vs. preferring to focus on more basic research, or alignment / preparation-for-AGI, etc.’)
You could try to get multiple of those properties at once by assuming specific things about the world’s overall adequacy and/or about the space of all reachable intelligent systems; but from Eliezer’s perspective these views fall somewhere on the spectrum between ‘unsupported speculation’ and ‘flatly contradicted by our observations so far’, and there are many ways to try to tweak civilization to be more adequate and/or the background CS facts to be more continuous, and still not hit the narrow target “a complete 4 year interval in which world output doubles” (before AGI destroys the world or a pivotal act occurs).
(I’m probably getting a bunch of details about Eliezer’s actual model wrong above, but my prediction is that his answer will at least roughly look like this.)
Affirmed.
Well said! This resonates with my Eliezer-model too.
Taking this into account I’d update my guess of Eliezer’s position to:
Eliezer: 5% soft takeoff, 80% hard takeoff, 15% something else
This last “something else” bucket added because “the Future is notoriously difficult to predict” (paraphrasing Eliezer).