[Haven’t read those fictions or the quoted spoilers]
Glad to hear you saying this. In fact, I’ve had a couple quite concerning conversations with a couple different people in which I’m like “so what would you do if you accidentally invented a Friendly AGI in your basement one day? what are some of your important first actions?” and they don’t give one of the incredibly obvious and important answers, and don’t even necessarily agree to it after I say it.
I’ll add that the HM attitude extends to various other things. For example, some people around here are contemptuous of bioethics. I can see where they’re coming from, but I think the attitude is quite wrong, and in particular, one does want to “bring in many stakeholders to the conversation as coequal voices”, because that’s how you be non-HM. Cf. https://www.lesswrong.com/posts/yH9FtLgPJxbimamKg/genomic-emancipation-contra-eugenics
one does want to “bring in many stakeholders to the conversation as coequal voices”, because that’s how you be non-HM
I think it’s quite shameful how rationalists aim to optimize the world and yet are generally uninterested in what life is like for many kinds of normal people, or what problems they face. To rule well, you should know what life is like for your subjects, and what they care about, at minimum.
But there’s also the pattern where “if your users tell you about a problem, they’re always right, and if they tell you the solution, they’re almost always wrong.” And many common forms of “consulting all the stakeholders” ends up giving a veto to special interests, which ends up strangling liberty.
There are some to my knowledge unsolved problems here.
To rule well, you should know what life is like for your subjects, and what they care about, at minimum.
This is still totally High Modernist. To actually rule well you should actually not try to rule as much and should actually try to share power, which includes giving up power. Not giving up all power, but, you know, you leave after your second term in office, and you create a parliament.
For example, some people around here are contemptuous of bioethics.
I do wonder how much of this is “contemptuous of bioethics done badly”. I can see the argument for “that means we need to do bioethics well, not continue to cede ground” but I do think it’s important to be honest about when fields are failing.
it’s important to be honest about when fields are failing.
I agree, naturally. I criticize bioethicists as a group, precisely because they are, for the most part, AFAICT, failing to lead on moral questions around reprogenetics.
However, I think one has to make multiple updates. That observation indicates that bioethicists may not have the right abilities or motivations or other properties (determination, grit, courage, sanity, wisdom, what have you), and on those grounds could be dismissed as individuals or even as a group; but it also indicates that the problems themselves are especially difficult. The latter is often underappreciated. To avoid “HM bioethics” you actually have to cede power in the discussions, and even logistically doing that is difficult (I mean, I don’t know how to do it; I don’t have a near-fully-satisfying theory of how to give proper / coequal decision weight to all the stakeholders who should have that).
I think part of the HM mindset is precisely reacting to “a bunch of other people are doing it bad and kinda punishing me for trying to do it better” with “actually I should just be in charge and not worry about the concerns those people talk about” rather than “some other group of people would have to figure out how to do it better”.
[Haven’t read those fictions or the quoted spoilers] Glad to hear you saying this. In fact, I’ve had a couple quite concerning conversations with a couple different people in which I’m like “so what would you do if you accidentally invented a Friendly AGI in your basement one day? what are some of your important first actions?” and they don’t give one of the incredibly obvious and important answers, and don’t even necessarily agree to it after I say it.
I’ll add that the HM attitude extends to various other things. For example, some people around here are contemptuous of bioethics. I can see where they’re coming from, but I think the attitude is quite wrong, and in particular, one does want to “bring in many stakeholders to the conversation as coequal voices”, because that’s how you be non-HM. Cf. https://www.lesswrong.com/posts/yH9FtLgPJxbimamKg/genomic-emancipation-contra-eugenics
I think it’s quite shameful how rationalists aim to optimize the world and yet are generally uninterested in what life is like for many kinds of normal people, or what problems they face. To rule well, you should know what life is like for your subjects, and what they care about, at minimum.
But there’s also the pattern where “if your users tell you about a problem, they’re always right, and if they tell you the solution, they’re almost always wrong.” And many common forms of “consulting all the stakeholders” ends up giving a veto to special interests, which ends up strangling liberty.
There are some to my knowledge unsolved problems here.
This is still totally High Modernist. To actually rule well you should actually not try to rule as much and should actually try to share power, which includes giving up power. Not giving up all power, but, you know, you leave after your second term in office, and you create a parliament.
I agree on all counts.
I do wonder how much of this is “contemptuous of bioethics done badly”. I can see the argument for “that means we need to do bioethics well, not continue to cede ground” but I do think it’s important to be honest about when fields are failing.
I agree, naturally. I criticize bioethicists as a group, precisely because they are, for the most part, AFAICT, failing to lead on moral questions around reprogenetics.
However, I think one has to make multiple updates. That observation indicates that bioethicists may not have the right abilities or motivations or other properties (determination, grit, courage, sanity, wisdom, what have you), and on those grounds could be dismissed as individuals or even as a group; but it also indicates that the problems themselves are especially difficult. The latter is often underappreciated. To avoid “HM bioethics” you actually have to cede power in the discussions, and even logistically doing that is difficult (I mean, I don’t know how to do it; I don’t have a near-fully-satisfying theory of how to give proper / coequal decision weight to all the stakeholders who should have that).
I think part of the HM mindset is precisely reacting to “a bunch of other people are doing it bad and kinda punishing me for trying to do it better” with “actually I should just be in charge and not worry about the concerns those people talk about” rather than “some other group of people would have to figure out how to do it better”.
HM?
High Modernist