Such a machine would rapidly become intelligent enough to take control of the internet, use robots to build itself new hardware, do science on a massive scale, invent new computing technology and energy sources, or achieve similar dominating goals.
This section sounds like Sci-fi, and might make some readers take the work as a whole less seriously. On the other hand, is is a pretty reasonable prediction (in terms of “how big an impact an AI could have”, even if the details are wrong), so down-toning it would be a bit dishonest.
I don’t know what the best strategy is to talk about such technical risks without sounding like a loon. I’m more in favor of a more cautious path of keeping such claims vague and general unless they are preceded by enough explanations that they don’t seem that outlandish. But I don’t know your audience as well as you do, I just think it’s an aspect that requires more care than when writing for LessWrong.
Vague and general claims about the future are more likely to be accurate than detailed claims, even though we might find detailed claims more believable—see Burdensome Details.
This section sounds like Sci-fi, and might make some readers take the work as a whole less seriously. On the other hand, is is a pretty reasonable prediction (in terms of “how big an impact an AI could have”, even if the details are wrong), so down-toning it would be a bit dishonest.
I don’t know what the best strategy is to talk about such technical risks without sounding like a loon. I’m more in favor of a more cautious path of keeping such claims vague and general unless they are preceded by enough explanations that they don’t seem that outlandish. But I don’t know your audience as well as you do, I just think it’s an aspect that requires more care than when writing for LessWrong.
Vague and general is never a good idea.
Vague and general claims about the future are more likely to be accurate than detailed claims, even though we might find detailed claims more believable—see Burdensome Details.
My objection was about the writing. See Williams.