I stopped reading at point two, which is not only obvious hogwash but has been used over and over again to rationalize (after the fact) stupid decisions throughout history.
There are in fact some possible good reasons to make irrational decisions (for instance, see Hofstadter’s discussion of the Prisoner’s Dilemma in Metamagical Themas). But the mere fact that an irrational decision in the past produced a good actual outcome does not change the fact that it was irrational, and therefore presumably stupid. One can only decide using the information available at the time. There is no alternative; not to decide is to decide on a default.
In what sense is point two obvious hogwash? The right decision is the one which maximizes utility. The rational decision is the one which maximizes expected utility. These aren’t the same in general because agents in general don’t have complete information about the consequences of their actions. None of this seems particularly controversial to me, and I don’t see how anything you’ve said is a counterargument against it.
Unfortunately, we cannot know with certainty what the right decision is. Thus, the best we can do is to try to make “rational” or “optimal” decisions based on our preferences and incomplete information.
Yudkowsky-style metaethics emphasizes that “goodness” (or here, “rightness”) is a property of actions that propagates from effects to causes, and this is consistent with the way the adjective “right” is used here in the FAQ.
I stopped reading at point two, which is not only obvious hogwash but has been used over and over again to rationalize (after the fact) stupid decisions throughout history.
If the basic premises of rational decision making are obviously hogwash to you and you are not trying to learn more then you are in the wrong place.
I stopped reading at point two, which is not only obvious hogwash but has been used over and over again to rationalize (after the fact) stupid decisions throughout history.
There are in fact some possible good reasons to make irrational decisions (for instance, see Hofstadter’s discussion of the Prisoner’s Dilemma in Metamagical Themas). But the mere fact that an irrational decision in the past produced a good actual outcome does not change the fact that it was irrational, and therefore presumably stupid. One can only decide using the information available at the time. There is no alternative; not to decide is to decide on a default.
In what sense is point two obvious hogwash? The right decision is the one which maximizes utility. The rational decision is the one which maximizes expected utility. These aren’t the same in general because agents in general don’t have complete information about the consequences of their actions. None of this seems particularly controversial to me, and I don’t see how anything you’ve said is a counterargument against it.
The last sentence of the section agrees with you:
Yudkowsky-style metaethics emphasizes that “goodness” (or here, “rightness”) is a property of actions that propagates from effects to causes, and this is consistent with the way the adjective “right” is used here in the FAQ.
Keep reading; it gets better.
If the basic premises of rational decision making are obviously hogwash to you and you are not trying to learn more then you are in the wrong place.