Pardon my ignorance in asking this stupid question:
Isn’t the mainstream position among decision theorists that defect is the optimal strategy in a one-shot prisoner’s dilemma game? Sure, we can say something like “a TDT-agent will cooperate with another TDT-agent.” But even a TDT-agent will defect if it doesn’t know the identity of the counter-party. And TDT isn’t completely formalized yet, right?
In short, I’m not sure of the value of snarking a popularizer of decision theory for using the word “rationality” differently than this community uses the word, particularly when our usage is (unfortunately) not the consensus definition in the relevant field.
Depending on the payoff scale, a TDT agent will cooperate if it believes that the other agent has some (high enough) chance of being a TDT agent. In other words, raise the sanity waterline high enough, and TDT cooperates.
TDT / superrationality will defect probabilistically given a high enough payoff for defection, even against a known-TDT agent.
In short: TDT and superrationality theories aren’t as simple as some here make them out to be, and the one-shot prisoner’s dilemma has hidden depths for smart players.
Of course, the rational thing to do is to convince everyone ELSE to be “superrational”, and convince them that you are ALSO “superrational”, and then defect if you actually play a prisoner’s dilemma for sufficiently high stakes.
Eliezer has done a good job of this. Hofstadter too. Inventing the term “superrationality” for “magicalthinking” was a good move.
Depending on the payoff scale, a TDT agent will cooperate if it believes that the other agent has some (high enough) chance of being a TDT agent.
He also has to believe that the other agent believes with sufficient confidence that he is a suitable kind of agent. Same population makeup considerations apply.
Increasing the rationality of a population can lead to that population being worse off, even by the “rationality as systematized winning” definition, if the members are at odds with each other. You don’t necessarily want your adversaries to be more rational. But the reason that part made me cringe is that it carries so many caveats that it’s incredibly misleading. There are many contexts where defecting in a prisoner’s dilemma is a predictably bad idea, and rationality already has image problems as being rigid and unadaptable as well as antisocial without reinforcing them.
The video probably couldn’t really have gone into a proper discussion of what variations of the prisoner’s dilemma make defection appropriate without breaking the flow, but it really wasn’t helped by a message of “paradoxically, rationality can leave you worse off.”
I agree with your point that usage of rationality this way exacerbates the problem of straw-vulcanism. I’m not sure the criticism is aimed at the lowest hanging fruit in this case, but your point is well taken.
Your point about the dangerousness of rationality is not on point. I am almost strictly better off in a society in which the members are not at odds than in one where they are at odds. But whether one of the exceptions to that statement applies does not appear to depends on how dangerous individual members are. Unless individuals are so dangerous that mutually assured destruction is an important part of the analysis. And that is currently so unrealistic (at the individual level) that I don’t think it is worth talking about.
Your point about the dangerousness of rationality is not on point.
My point is not that rationality is dangerous. It’s possible to formulate situations in which increasing the rationality of a population leaves the population worse off, rather like it’s possible to formulate situations that reward agents for having priors that would normally be stupid. The fact that you can contrive such situations doesn’t constitute a compelling criticism of rationality, which is what makes the video problematic. If rationality really were that dangerous, the implicit criticism would be entirely valid.
Pardon my ignorance in asking this stupid question:
Isn’t the mainstream position among decision theorists that defect is the optimal strategy in a one-shot prisoner’s dilemma game? Sure, we can say something like “a TDT-agent will cooperate with another TDT-agent.” But even a TDT-agent will defect if it doesn’t know the identity of the counter-party. And TDT isn’t completely formalized yet, right?
In short, I’m not sure of the value of snarking a popularizer of decision theory for using the word “rationality” differently than this community uses the word, particularly when our usage is (unfortunately) not the consensus definition in the relevant field.
Depending on the payoff scale, a TDT agent will cooperate if it believes that the other agent has some (high enough) chance of being a TDT agent. In other words, raise the sanity waterline high enough, and TDT cooperates.
TDT / superrationality will defect probabilistically given a high enough payoff for defection, even against a known-TDT agent.
In short: TDT and superrationality theories aren’t as simple as some here make them out to be, and the one-shot prisoner’s dilemma has hidden depths for smart players.
Of course, the rational thing to do is to convince everyone ELSE to be “superrational”, and convince them that you are ALSO “superrational”, and then defect if you actually play a prisoner’s dilemma for sufficiently high stakes.
Eliezer has done a good job of this. Hofstadter too. Inventing the term “superrationality” for “magicalthinking” was a good move.
He also has to believe that the other agent believes with sufficient confidence that he is a suitable kind of agent. Same population makeup considerations apply.
Increasing the rationality of a population can lead to that population being worse off, even by the “rationality as systematized winning” definition, if the members are at odds with each other. You don’t necessarily want your adversaries to be more rational. But the reason that part made me cringe is that it carries so many caveats that it’s incredibly misleading. There are many contexts where defecting in a prisoner’s dilemma is a predictably bad idea, and rationality already has image problems as being rigid and unadaptable as well as antisocial without reinforcing them.
The video probably couldn’t really have gone into a proper discussion of what variations of the prisoner’s dilemma make defection appropriate without breaking the flow, but it really wasn’t helped by a message of “paradoxically, rationality can leave you worse off.”
I agree with your point that usage of rationality this way exacerbates the problem of straw-vulcanism. I’m not sure the criticism is aimed at the lowest hanging fruit in this case, but your point is well taken.
Your point about the dangerousness of rationality is not on point. I am almost strictly better off in a society in which the members are not at odds than in one where they are at odds. But whether one of the exceptions to that statement applies does not appear to depends on how dangerous individual members are. Unless individuals are so dangerous that mutually assured destruction is an important part of the analysis. And that is currently so unrealistic (at the individual level) that I don’t think it is worth talking about.
My point is not that rationality is dangerous. It’s possible to formulate situations in which increasing the rationality of a population leaves the population worse off, rather like it’s possible to formulate situations that reward agents for having priors that would normally be stupid. The fact that you can contrive such situations doesn’t constitute a compelling criticism of rationality, which is what makes the video problematic. If rationality really were that dangerous, the implicit criticism would be entirely valid.