At some point in the development of AI, there will be a very swift increase in the optimization power of the most powerful AI, moving from a non-dangerous level to a level of superintelligence. (Fast takeoff)
...unless people want it to go slowly. It isn’t a law of nature that things will go quickly. It seems likely that a more unified society will be able to progress as slowly as it wants to. There are plenty of proposals to throttle development—via “nannies” or other kinds of safety valve.
IMO, the “rapid takeoff” idea should probably be seen as a fundraising ploy. It’s big, scary, and it could conceivably happen—just the kind of thing for stimulating donations.
IMO, the “rapid takeoff” idea should probably be seen as a fundraising ploy. It’s big, scary, and it could conceivably happen—just the kind of thing for stimulating donations.
It seems that SIAI would have more effective methods for fundraising, e.g. simply capitalizing on “Rah Singularity!”. I therefore find this objection somewhat implausible.
See Singularity University and the Singularity Summit. Cheerleading is more effective in raising money (and there are many more things to do in that line), but money to cheerlead and accelerate, not to try to shape outcomes.
...unless people want it to go slowly. It isn’t a law of nature that things will go quickly. It seems likely that a more unified society will be able to progress as slowly as it wants to. There are plenty of proposals to throttle development—via “nannies” or other kinds of safety valve.
Insistence on a rapid takeoff arises from a position of technological determinism. It ignores sociological factors.
IMO, the “rapid takeoff” idea should probably be seen as a fundraising ploy. It’s big, scary, and it could conceivably happen—just the kind of thing for stimulating donations.
It seems that SIAI would have more effective methods for fundraising, e.g. simply capitalizing on “Rah Singularity!”. I therefore find this objection somewhat implausible.
They did originally try the “Rah Singularity” strategy. Only more recently did they switch to using more negative marketing.
See Singularity University and the Singularity Summit. Cheerleading is more effective in raising money (and there are many more things to do in that line), but money to cheerlead and accelerate, not to try to shape outcomes.