“Even intrapersonal ethics can help protect you from black swans and fundamental mistakes. That logic doesn’t change its structure when you double the value of the stakes, or even raise them to the level of a world.”
I’m not so sure. The kind of deontological ethics that you are talking about works well in human social interactions. When removed from that context, why do you think that it will still work?
For example, Knut Haukelid broke a deontological rule in order to make a significant dent in the subjective probability (given his knowledge) of the the nazis winning WWII. I think that he did the right thing, and I think that in such an extreme case one ought to act according to the greater good.
The problem is working out when one is in a sufficiently extreme case. For the readers of Overcoming Bias, and those interested in the singularity, this is a tough question. Clever men, such as yourself, tell us that the fate of the entire human race rests upon solving the FAI problem. Does this count as extreme? Does it count as extreme enough to justify damaging one’s personal life, one’s friends or family?
My answer to such questions of “greater good” versus “duty” used to be to favor the former, but my experiences in life have shown me that it is better to try to avoid such choices. Looking back on the times when I have stuck by my friends or my duties to my disadvantage, and the times where I have betrayed people or lied, (Yes, I have done both several times), I realize that in every single case there was a third option available if I had just thought about the problem clearly enough.
“Even intrapersonal ethics can help protect you from black swans and fundamental mistakes. That logic doesn’t change its structure when you double the value of the stakes, or even raise them to the level of a world.”
I’m not so sure. The kind of deontological ethics that you are talking about works well in human social interactions. When removed from that context, why do you think that it will still work?
For example, Knut Haukelid broke a deontological rule in order to make a significant dent in the subjective probability (given his knowledge) of the the nazis winning WWII. I think that he did the right thing, and I think that in such an extreme case one ought to act according to the greater good.
The problem is working out when one is in a sufficiently extreme case. For the readers of Overcoming Bias, and those interested in the singularity, this is a tough question. Clever men, such as yourself, tell us that the fate of the entire human race rests upon solving the FAI problem. Does this count as extreme? Does it count as extreme enough to justify damaging one’s personal life, one’s friends or family?
My answer to such questions of “greater good” versus “duty” used to be to favor the former, but my experiences in life have shown me that it is better to try to avoid such choices. Looking back on the times when I have stuck by my friends or my duties to my disadvantage, and the times where I have betrayed people or lied, (Yes, I have done both several times), I realize that in every single case there was a third option available if I had just thought about the problem clearly enough.