A Bayesian superintelligence? There is no natural example. The development could require as much resources as fusion and anti-aging combined. Or maybe such an intelligence is not possible at all.
This seems like a strawman. An AI that could do everything an average human could do but ten times as fast would already have unbelievable ramifications on the economy. And one which was as competent as the most competent existing human, but 100 times as fast and instantly clonable would definitely be sufficiently powerful to require aligning it correctly. Whether or not it’s a Bayesian superintelligence seems kind of irrelevant.
I agree, I failed to sufficiently steelman this argument. How would you improve it?
I also agree with you on that a human-like AI could become a Transformative AI. Maybe even speed-ups are not required for that, and the easy cloning will suffice. Moreover, an AI that can perform only a small subset of human tasks could become a TAI.
This seems like a strawman. An AI that could do everything an average human could do but ten times as fast would already have unbelievable ramifications on the economy. And one which was as competent as the most competent existing human, but 100 times as fast and instantly clonable would definitely be sufficiently powerful to require aligning it correctly. Whether or not it’s a Bayesian superintelligence seems kind of irrelevant.
I agree, I failed to sufficiently steelman this argument. How would you improve it?
I also agree with you on that a human-like AI could become a Transformative AI. Maybe even speed-ups are not required for that, and the easy cloning will suffice. Moreover, an AI that can perform only a small subset of human tasks could become a TAI.