Distinguishing CDT from FDT/TDT in intuitive cases tends to be a lot harder than it looks. And I think it’s important to be extremely careful about what we categorize as CDT+being clever versus FDT/TDT. My impression is that this story more often frequently the former.
At first, the population was composed of a humble race of agents called the ceedeetee. When two of the ceedeetee met each other, each would name the number 5, and receive a payoff of 5, and all was well.
I’m not sure it’s obvious that all ceedeetee will meet five when they meet each other.
In an environment where there is zero information, this would be true (ie guessing >5 causes the gueesser to get outcompeted by those who will miss fewer payoffs and guessing less causes them to get outcompeted genetically by their partners in the game) but it’s clearly not true in this particular context. Instead, it seems more likely that ceedeetrees will on-net guess (and get) five based on whether their analysis of their partner tells them what they can get away with (ie A scares B so B only offers 4 and B offers 6, but B scares C so B offers 6 and C offers 4, but C scares A and so on...). I’d expect an equlibrium that’s suboptimal but has cyclical relationships between the participants.
Since output from the game determines evolutionary fitness, any ceedeetees who get some payoffs from other sources (ie this guy I just met seems nice but that other guy didn’t so I’m gonna give a 4 to this guy and a 6 to the other guy) won’t always output five.
These points are kind of pedantic but it’s importance to notice, if this happens, nine-bots get destroyed. They always guess way too high and the inherent noise in how a population of actual ceedeetee play the game will be hard to recover from.
Then one day, a simple race of 9-bots invaded the land. The 9-bots would always name the number 9!
Where exactly would we expect the 9-bots to come from? If they were all trapped on a ship together, they would’ve just continously lost the game until they died Again, this is kind of pedantic but, as you point out, the population distributions matter.
And from that day onward, whenever Funk-tunul met a fellow ceedeetee agent—if “fellow” is the right word here, which it isn’t—she would announce that she was going to name 9, and do so. And though the ceedeetee agents’ output channels would light up with the standard inidicators of outrage and betrayal, they would reason causally, and name 1.
A very key part of what Funk-tunul is doing here is telling the ceedeetee agents beforehand that she’ll say nine. Again, it strikes me that, if a ceeteedee noticed they could cause their partners to guess numbers lower than five, they definitely would do that. Funk-tunul isn’t winning because of a better decision theory here; she’s winning because she’s more clever. at manipulating other ceedeetee.
However, in real life, this implies that Funk-tunul would not be successful. A ceedeetee would’ve, in the past, tried to credibly show that they always say nine until the population equilibrates to having a defense mechanism against this particular action.
They reasoned: suppose the fraction of ceedeetee agents in the population is p, the fraction of funk-tunul agents is q, and the fraction of 9-bots is 1−p−q. If we establish a policy of submitting to the 9-bots’ extortion, we’ll have an average payoff of 9p+5q+1⋅(1−p−q)=8p+4q+1 and the 9-bots will have an average payoff of 9p+9q. If we defy the 9-bots while continuing to extort our ceedeetee cousins, we’ll have an average payoff of 9p+5q, whereas the 9-bots will have an average payoff of 9p. Whether it’s better to submit or defy depends on the values of p and q. It’s not obviously possible for defiance to be the right choice given what we know, but if we can coordinate to meet fellow funk-tunul agents more often—if we drop the assumption of uniform random encounters—the calculus changes …
This doesn’t strike me as acausal reasoning; just long-termist reasoning. Given the (presumably exponential) population dynamics, a ceedeetee could easily predict that letting the nine-bot get nine points would help that nine-bot reproduce more nine-bots. If ceedeetee’rs are in the game to maximize fitness as opposed to utility, they’ll definitely establish a norm against helping nine-bots to protect against the exponential cost that nine-bots will have for the future. If they’re in the game to maximize their points in the game, this isn’t true (they’ll just defect against the future) but funk-tunul’s reasoning suggests that this isn’t what’s going on.
It’s not obviously possible for defiance to be the right choice given what we know, but if we can coordinate to meet fellow funk-tunul agents more often—if we drop the assumption of uniform random encounters—the calculus changes …
If we drop limiting assumptions once funk-tunul agents get involves, it seems pretty clear that the funk-tunul agents will do better than the ceedeetee previously did.
Before the two agents could name their numbers, Graddes spoke. “Please. Why are you doing this?” she pleaded. “I can’t hate the 9-bots for their extortion, for they are a simple race and could not do otherwise. But you—we’re cousins. Your lineage is a fork of mine. You know it’s not fair for your people to always name the number 9 when meeting mine. Yet you do so anyway, knowing that we have no choice but to name the number 1 if we want any payoff at all. Why?”
“Don’t hate the player,” said Tim’liss, her output channels dimming and brightening in a interpolated pattern one-third of the way between the standard indicators for sympathy and contempt. “Hate life.”
We just dropped the random-interaction assumption. Why don’t the ceedeetee just only interacting with fellow ceedeetee? Choosing only to interact with ceedeetee would get them waaaaay more points.
Also, this is evidence that the ceedeetee in the game care about stuff beyond just the scores they get in the game and reenforces my point that the events as-described don’t really make sense in evolutionary setting. Given this, it’s worth pointing out is that the actual thing Tim’liss is doing here is supporting a race to the bottom that optimizes only reproductive fitness. Engaging in a race to the bottom for reproductive fitness is Not Good timeless decision theory.
Distinguishing CDT from FDT/TDT in intuitive cases tends to be a lot harder than it looks. And I think it’s important to be extremely careful about what we categorize as CDT+being clever versus FDT/TDT. My impression is that this story more often frequently the former.
I’m not sure it’s obvious that all ceedeetee will meet five when they meet each other.
In an environment where there is zero information, this would be true (ie guessing >5 causes the gueesser to get outcompeted by those who will miss fewer payoffs and guessing less causes them to get outcompeted genetically by their partners in the game) but it’s clearly not true in this particular context. Instead, it seems more likely that ceedeetrees will on-net guess (and get) five based on whether their analysis of their partner tells them what they can get away with (ie A scares B so B only offers 4 and B offers 6, but B scares C so B offers 6 and C offers 4, but C scares A and so on...). I’d expect an equlibrium that’s suboptimal but has cyclical relationships between the participants.
Since output from the game determines evolutionary fitness, any ceedeetees who get some payoffs from other sources (ie this guy I just met seems nice but that other guy didn’t so I’m gonna give a 4 to this guy and a 6 to the other guy) won’t always output five.
These points are kind of pedantic but it’s importance to notice, if this happens, nine-bots get destroyed. They always guess way too high and the inherent noise in how a population of actual ceedeetee play the game will be hard to recover from.
Where exactly would we expect the 9-bots to come from? If they were all trapped on a ship together, they would’ve just continously lost the game until they died Again, this is kind of pedantic but, as you point out, the population distributions matter.
A very key part of what Funk-tunul is doing here is telling the ceedeetee agents beforehand that she’ll say nine. Again, it strikes me that, if a ceeteedee noticed they could cause their partners to guess numbers lower than five, they definitely would do that. Funk-tunul isn’t winning because of a better decision theory here; she’s winning because she’s more clever. at manipulating other ceedeetee.
However, in real life, this implies that Funk-tunul would not be successful. A ceedeetee would’ve, in the past, tried to credibly show that they always say nine until the population equilibrates to having a defense mechanism against this particular action.
This doesn’t strike me as acausal reasoning; just long-termist reasoning. Given the (presumably exponential) population dynamics, a ceedeetee could easily predict that letting the nine-bot get nine points would help that nine-bot reproduce more nine-bots. If ceedeetee’rs are in the game to maximize fitness as opposed to utility, they’ll definitely establish a norm against helping nine-bots to protect against the exponential cost that nine-bots will have for the future. If they’re in the game to maximize their points in the game, this isn’t true (they’ll just defect against the future) but funk-tunul’s reasoning suggests that this isn’t what’s going on.
If we drop limiting assumptions once funk-tunul agents get involves, it seems pretty clear that the funk-tunul agents will do better than the ceedeetee previously did.
We just dropped the random-interaction assumption. Why don’t the ceedeetee just only interacting with fellow ceedeetee? Choosing only to interact with ceedeetee would get them waaaaay more points.
Also, this is evidence that the ceedeetee in the game care about stuff beyond just the scores they get in the game and reenforces my point that the events as-described don’t really make sense in evolutionary setting. Given this, it’s worth pointing out is that the actual thing Tim’liss is doing here is supporting a race to the bottom that optimizes only reproductive fitness. Engaging in a race to the bottom for reproductive fitness is Not Good timeless decision theory.