Why select a deterministic game with complete information for this? I suspect games like poker or backgammon would be easier for the adversarial advisors to fool the player and that these games are a better model of the real world scenario.
I’m not sure about poker, but I think for backgammon it’d be harder to get three levels where C beats B beats A reliably. I’m not a backgammon expert, but I could win games against experts—it’s enough to be competent and lucky. A may also learn too fast—becoming competent is much faster for backgammon than for chess. (needing a larger sample size due to randomness makes A learning more of a problem—this may apply with poker too??)
I have a lot more experience and skill at chess, but it’s still pretty simple to find players who’ll beat me 90% of the time.
[...] the corresponding winning probability of a player who is exactly one standard deviation better than his opponent. We refer to this probability as p^sd . For comparison, we also provide the winning probablities when a 99% percentile player is matched against a 1% percentile player, which we call p99 1 .
Go & Chess (p^sd=83.3,72.9) are notably above Backgammon (p^sd=53.6%)
Agreed that it could be a bit more realistic that way, but the main constraint here is that we need a game where there are three distinct levels of players who always beat each other. The element of luck in games like poker and backgammon makes that harder to guarantee (as suggested by the stats Joern_Stoller brought up). And another issue is that it’ll be harder to find a lot of skilled players at different levels from any game that isn’t as popular as chess is—even if we find an obscure game that would in theory be a better fit for the experiment, we won’t be able to find any Cs for it.
Why select a deterministic game with complete information for this? I suspect games like poker or backgammon would be easier for the adversarial advisors to fool the player and that these games are a better model of the real world scenario.
For an entertainingly thematic choice, I’d recommend Twilight Struggle.
I’m not sure about poker, but I think for backgammon it’d be harder to get three levels where C beats B beats A reliably. I’m not a backgammon expert, but I could win games against experts—it’s enough to be competent and lucky. A may also learn too fast—becoming competent is much faster for backgammon than for chess. (needing a larger sample size due to randomness makes A learning more of a problem—this may apply with poker too??)
I have a lot more experience and skill at chess, but it’s still pretty simple to find players who’ll beat me 90% of the time.
See Table 2 in https://www.emilkirkegaard.com/p/skill-vs-luck-in-games for
Go & Chess (p^sd=83.3,72.9) are notably above Backgammon (p^sd=53.6%)
Oh that’s cool—nice that someone’s run the numbers on this.
I’m actually surprised quite how close-to-50% both backgammon and poker are.
Agreed that it could be a bit more realistic that way, but the main constraint here is that we need a game where there are three distinct levels of players who always beat each other. The element of luck in games like poker and backgammon makes that harder to guarantee (as suggested by the stats Joern_Stoller brought up). And another issue is that it’ll be harder to find a lot of skilled players at different levels from any game that isn’t as popular as chess is—even if we find an obscure game that would in theory be a better fit for the experiment, we won’t be able to find any Cs for it.