Also there is no consensus on what is a vital social norm. I think we can agree that common random killing sprees would perhaps fall into this. But even that in itself if surrounded by the right memetic scaffolding could be made to work (perhaps a “kill the guy you hate day” could work). Social stability and prosperity is mostly polygenic.
Kill likely to succeed AGI creators who haven’t created a sane goal system (when no other means will work to stop them). Although I know Tim doesn’t accept even that exception.
Me? Yes, those who go on a programmer-killing spree are unlikely to be viewed favourably by me. I don’t think all murder is impossible to justify—but prospective killers would need to be really, really convincing about their motivation in order to avoid being black-balled by the rest of society.
“I have this paranoid fantasy about their mechanical offspring taking over the world”—is the type of thing that would not normally be regarded as adequate justification for killing people.
How many, over the decades, have fallen under “likely to succeed”? e.g. according to scientists/”experts”, investors, project leaders, etc. Whose estimate gets used, anyway?
How many, over the decades, have fallen under “likely to succeed”?
None.
e.g. according to scientists/”experts”, investors, project leaders, etc. Whose estimate gets used, anyway?
Whoever is making the decision. That’s how decisions work. Said person would use whatever information is relevant to them. They will then decide whether they need to take action to prevent the destruction of all things good and light or whether they will take action to prevent someone who they believe to be intending to kill due to paranoia.
Kill likely to succeed AGI creators who haven’t created a sane goal system (when no other means will work to stop them). Although I know Tim doesn’t accept even that exception.
Me? Yes, those who go on a programmer-killing spree are unlikely to be viewed favourably by me. I don’t think all murder is impossible to justify—but prospective killers would need to be really, really convincing about their motivation in order to avoid being black-balled by the rest of society.
“I have this paranoid fantasy about their mechanical offspring taking over the world”—is the type of thing that would not normally be regarded as adequate justification for killing people.
How many, over the decades, have fallen under “likely to succeed”? e.g. according to scientists/”experts”, investors, project leaders, etc. Whose estimate gets used, anyway?
None.
Whoever is making the decision. That’s how decisions work. Said person would use whatever information is relevant to them. They will then decide whether they need to take action to prevent the destruction of all things good and light or whether they will take action to prevent someone who they believe to be intending to kill due to paranoia.