Powerful machine intelligences can be expected to have natural drives to eliminate competing goal-based systems.
So, unless there are safeguards against it, a machine intelligence is likely to assassinate potential designers of other machine intelligences which may have subtly different goals. IMO, assassinating your competitors is not an acceptable business practice.
CEV doesn’t seem to have much in the way of safeguards against this. It isn’t even constrained to follow the law of the land. I think as it stands, it has clear criminal tendencies—and so should not be built.
Have you ever heard of the last president of the US? He’s a particularly extreme example in criminality for a president, but I’m pretty sure that all or nearly all presidents would count extremely criminal compared to what you are used to from day-to-day life. Congressman likewise.
What sense of ‘criminal’ are you using here? Presumably not ‘convicted of a crime by a court’ since that is relatively rare for politicians. Do you mean ‘have committed acts that are against the law but have not been prosecuted’ or do you mean ‘have committed acts that in my view are/should-be-viewed-as criminal but have not actually broken the law technically’?
He has publicly admitted to ordering violations of the FISA statute, a felony, so certainly “have committed acts that are against the law but have not been prosecuted”.
US politics is not my area—but I don’t think there has ever been a criminal prosecution of an incumbent president.
However, sometimes criminals do get some influence and cause significant damage. It seems like a good reason to do what you can to prevent such things from happening.
Powerful machine intelligences can be expected to have natural drives to eliminate competing goal-based systems.
So, unless there are safeguards against it, a machine intelligence is likely to assassinate potential designers of other machine intelligences which may have subtly different goals. IMO, assassinating your competitors is not an acceptable business practice.
CEV doesn’t seem to have much in the way of safeguards against this. It isn’t even constrained to follow the law of the land. I think as it stands, it has clear criminal tendencies—and so should not be built.
People aren’t constrained to follow the law of the land either.
Fortunately for the rest of us, most criminals are relatively impotent.
Have you ever heard of the last president of the US? He’s a particularly extreme example in criminality for a president, but I’m pretty sure that all or nearly all presidents would count extremely criminal compared to what you are used to from day-to-day life. Congressman likewise.
Hence the qualifier “most”.
Also, does driving faster than the speed limit make you technically a criminal? How about downloading pirate MP3s?
What sense of ‘criminal’ are you using here? Presumably not ‘convicted of a crime by a court’ since that is relatively rare for politicians. Do you mean ‘have committed acts that are against the law but have not been prosecuted’ or do you mean ‘have committed acts that in my view are/should-be-viewed-as criminal but have not actually broken the law technically’?
He has publicly admitted to ordering violations of the FISA statute, a felony, so certainly “have committed acts that are against the law but have not been prosecuted”.
US politics is not my area—but I don’t think there has ever been a criminal prosecution of an incumbent president.
However, sometimes criminals do get some influence and cause significant damage. It seems like a good reason to do what you can to prevent such things from happening.