Environmentalists care about direct and specific threats. The rest of them seem to care about possible loss of something we call “Friendliness” here.
HBD people believe that different cultures encourage different behaviors, which creates different selection pressures, and the differences gradually get encoded in genes. Which means that people from other cultures are for us like 99% humans + 1% Pebblesorters. We should optimize for our values. -- In addition to this, there are also direct specific threats, e.g. violence.
Leftists point out that the Invisible Hand of Market is not Friendly, therefore it may optimize for things we consider horrible; just like evolution, and for the same reason. This part seems obvious to me; I only object connotationally that institutions in general are not Friendly, including the institutions created by leftists.
Catholics have a model of the world where Friendliness comes from God, and humans themselves are not Friendly. We are broken; we don’t automatically optimize for things we actualy like to have. Azathoth is in us. Ignoring God’s commands—and sexual behavior is where doing so is most tempting—means giving up the Friendliness and letting Azathoth optimize for its own purposes.
LessWrongians point out that an AI could be a powerful threat to Friendliness, that it is likely to be the threat, and that it may be too powerful to be stopped once this becomes obvious.
To sum up the differences: Catholics pretend they have an external source of Friendliness and try to preserve the connection with it. HBD people worry about external sources of Unfriendliness. Leftists worry about existing forces that slowly but persistently optimize away from Friendliness. LessWrongians worry about a new very fast and powerful source we may create.
My opinion is that Friendliness is fragile and in danger. Azathoth and the hypothetical Unfriendly AI will optimize away from it. There is no equivalent power optimizing towards it; we only have inertia on our side. Unless we develop the Friendly AI or become greatly more rational ourselves (unless either MIRI or CFAR reaches their goal), we are doomed in long term. But of course some other disaster can destroy us even faster; we should not forget about immediate threats.
I think all these groups except for Catholics have a valid point (and even Catholics have some useful heuristics; the problem is that their other heuristics are harmful, and their model is hopelessly wrong) and I have no idea how to evaluate which one deserves most attention. My reason to choose LW is that it is the smallest group, so its concerns are most likely to be ignored. Also, in long term only LW has a satisfying solution (although this itself does not prove that someone else shouldn’t get priority in short term), the other groups are merely slowing down the inevitable.
Environmentalists care about direct and specific threats. The rest of them seem to care about possible loss of something we call “Friendliness” here.
HBD people believe that different cultures encourage different behaviors, which creates different selection pressures, and the differences gradually get encoded in genes. Which means that people from other cultures are for us like 99% humans + 1% Pebblesorters. We should optimize for our values. -- In addition to this, there are also direct specific threats, e.g. violence.
Leftists point out that the Invisible Hand of Market is not Friendly, therefore it may optimize for things we consider horrible; just like evolution, and for the same reason. This part seems obvious to me; I only object connotationally that institutions in general are not Friendly, including the institutions created by leftists.
Catholics have a model of the world where Friendliness comes from God, and humans themselves are not Friendly. We are broken; we don’t automatically optimize for things we actualy like to have. Azathoth is in us. Ignoring God’s commands—and sexual behavior is where doing so is most tempting—means giving up the Friendliness and letting Azathoth optimize for its own purposes.
LessWrongians point out that an AI could be a powerful threat to Friendliness, that it is likely to be the threat, and that it may be too powerful to be stopped once this becomes obvious.
To sum up the differences: Catholics pretend they have an external source of Friendliness and try to preserve the connection with it. HBD people worry about external sources of Unfriendliness. Leftists worry about existing forces that slowly but persistently optimize away from Friendliness. LessWrongians worry about a new very fast and powerful source we may create.
My opinion is that Friendliness is fragile and in danger. Azathoth and the hypothetical Unfriendly AI will optimize away from it. There is no equivalent power optimizing towards it; we only have inertia on our side. Unless we develop the Friendly AI or become greatly more rational ourselves (unless either MIRI or CFAR reaches their goal), we are doomed in long term. But of course some other disaster can destroy us even faster; we should not forget about immediate threats.
I think all these groups except for Catholics have a valid point (and even Catholics have some useful heuristics; the problem is that their other heuristics are harmful, and their model is hopelessly wrong) and I have no idea how to evaluate which one deserves most attention. My reason to choose LW is that it is the smallest group, so its concerns are most likely to be ignored. Also, in long term only LW has a satisfying solution (although this itself does not prove that someone else shouldn’t get priority in short term), the other groups are merely slowing down the inevitable.