An AI could screw us up just by giving bad advice. We’ll be likely to trust it, because it’s smart and we’re too lazy to think. A modern GPS receiver can make you drive into a lake. An evil AI could ruin companies, start wars, or create an evil robot without lifting a finger.
Besides, it’s more fun to create FAI and let it do what it wants than to build Skynet and then try to confine it forever. You’ll still have only one chance to test it, whenever you decide to do that.
An AI could screw us up just by giving bad advice. We’ll be likely to trust it, because it’s smart and we’re too lazy to think. A modern GPS receiver can make you drive into a lake. An evil AI could ruin companies, start wars, or create an evil robot without lifting a finger.
Besides, it’s more fun to create FAI and let it do what it wants than to build Skynet and then try to confine it forever. You’ll still have only one chance to test it, whenever you decide to do that.