Only if one of them is diagonalizing the other (acting contrary to what the other would’ve predicted about its actions). If this isn’t happening, maybe there is no problem.
Ah, yes, of course. I’ll update the description.
Agree on all the rest, I think. I didn’t intent to establish a strict ordering of agents (though my usage of bigger and smaller in the strict case of adversarial diagonalizing agents sure suggested it). In those case I find it a useful visualization to think about bigger and smaller.
It’s from Cantor’s diagonal argument.
I agree that “diagonalization” is a fine term for the specific narrow thing where you choose actions contrary to what the other agent would have predicted you would do, in the way described here, but I am more talking about the broader phenomenon of “simulating other agents adversarially in order to circumvent their predictions”. “Leveling” is apparently a term from poker that means something kind of similar and more general:
Levelling
Leveling in poker is the process of anticipating what your opponent thinks you are thinking, often leading to deeper layers of strategic decision-making.
Its purpose is to outthink opponents by operating on a higher mental “level” than they are.
Like, I would like a term for this kind of thing that is less opinionated about the exact setup, and technical limitations. Like, I am pretty sure there is a more general phenomenon here.
I am more talking about the broader phenomenon of “simulating other agents adversarially in order to circumvent their predictions”
The idea of “simulating adversarially” might be a bit confusing in the context of diagonalization, since it’s the diagonalization that is adversarial, not the simulation. In particular, you’d want mutual simulation (or rather more abstract reasoning) for coordination. If you merely succeed in acting contrary to a prediction, making the prediction wrong, that’s not diagonalization. What diagonalization does is make the prediction not-happen in the first place (or in the case of putting a credence on something, for the credence to remain at some weaker prior). So diagonalization is something done against a predictor whose prediction is targeted, rather than something done by the predictor. A diagonalizer might itself want to be a predictor, but that is not necessary if the prediction is just given to it.
Ah, yes, of course. I’ll update the description.
Agree on all the rest, I think. I didn’t intent to establish a strict ordering of agents (though my usage of bigger and smaller in the strict case of adversarial diagonalizing agents sure suggested it). In those case I find it a useful visualization to think about bigger and smaller.
I agree that “diagonalization” is a fine term for the specific narrow thing where you choose actions contrary to what the other agent would have predicted you would do, in the way described here, but I am more talking about the broader phenomenon of “simulating other agents adversarially in order to circumvent their predictions”. “Leveling” is apparently a term from poker that means something kind of similar and more general:
Like, I would like a term for this kind of thing that is less opinionated about the exact setup, and technical limitations. Like, I am pretty sure there is a more general phenomenon here.
The idea of “simulating adversarially” might be a bit confusing in the context of diagonalization, since it’s the diagonalization that is adversarial, not the simulation. In particular, you’d want mutual simulation (or rather more abstract reasoning) for coordination. If you merely succeed in acting contrary to a prediction, making the prediction wrong, that’s not diagonalization. What diagonalization does is make the prediction not-happen in the first place (or in the case of putting a credence on something, for the credence to remain at some weaker prior). So diagonalization is something done against a predictor whose prediction is targeted, rather than something done by the predictor. A diagonalizer might itself want to be a predictor, but that is not necessary if the prediction is just given to it.