Analogies can be found in many places. FDA prevents you from selling certain kinds of food? Sounds similar to ancient priests declaring food taboos for their followers. Vaccination? That’s just modern people performing a ritual to literally protect them from invisible threats. They even believe that a bad thing will happen to them if someone else in their neighborhood refuses to perform the ritual properly.
The difference is that we already have examples of food poisoning or people dying from a disease, but we do not have an example of a super-intelligent AI exterminating the humanity. That is a fair objection, but it is also clear why waiting to get the example first might be a wrong approach, so...
One possible approach is to look at smaller versions. What is a smaller version of “a super-intelligent AI exterminating the humanity”? If it is “a stupid program doing things its authors clearly did not intend”, then every software developer has stories to tell.
This is not the full answer, of course, but I think that a reasonable debate should be more like this.
Analogies can be found in many places. FDA prevents you from selling certain kinds of food? Sounds similar to ancient priests declaring food taboos for their followers. Vaccination? That’s just modern people performing a ritual to literally protect them from invisible threats. They even believe that a bad thing will happen to them if someone else in their neighborhood refuses to perform the ritual properly.
The difference is that we already have examples of food poisoning or people dying from a disease, but we do not have an example of a super-intelligent AI exterminating the humanity. That is a fair objection, but it is also clear why waiting to get the example first might be a wrong approach, so...
One possible approach is to look at smaller versions. What is a smaller version of “a super-intelligent AI exterminating the humanity”? If it is “a stupid program doing things its authors clearly did not intend”, then every software developer has stories to tell.
This is not the full answer, of course, but I think that a reasonable debate should be more like this.