We would all like to believe that if we win something it is because of our skill, but if we lose it is because of the good luck or cheating of our opponent. This is nowhere more apparent than in the game of backgammon. A good opponent will play in such a way that the dice rolls having highest probability will be “good rolls” for them on their next turn, enabling them to take the other player’s pieces, consolidate their position, etc. However, since there is an element of randomness, when these rolls actually come up it is very difficult even for me to believe that my opponent’s fortune is due to their skill rather than their luck. Whenever I am being consistently beaten by an opponent I have to fight very hard to make myself believe that there is actually something wrong with my play and figure out how to correct it.
This phenomenon is nowhere more apparent than in computer backgammon. Computer backgammon is essentially a solved problem. Unlike Chess or Go, the best neural network algorithms that run even on modest hardware will consistently beat most human players, and perhaps draw with the best in the world. In fact, we have learnt rather a lot from these algorithms. Many rules of thumb that were believed for years by most decent players have been overturned by computer simulations. Computers tend to play backgammon markedly more aggressively than is natural for a human, but we can show that they are nevertheless doing the right thing. If you are not an extremely high level player, playing against a good computer algorithm feels weird if you are used to playing against humans.
One of the most annoying results of this is that it is impossible to obtain a reliable user generated rating of a backgammon program. For example, GNU Backgammon is one of the two or three best algorithms in the world and an under-appreciated gem of the GNU project as a whole. A full 50% of the user reviews on the Ubuntu software centre accuse the program of cheating by fixing the dice rolls and give it a 1 or 2 star review. It is hilarious how many of these reviews include comments like they “know how to play” or have been “playing for a long time” as if this were evidence to support their claim. The rest of the reviews are by people who know better and they all give it 4 or 5 stars (4 is reasonable due to a few user interface quirks, but the algorithm itself is worth a 5). As a result, the averaged rating is only 3 stars, even though this is widely acknowledged as one of the three best backgammon programs in the world. It doesn’t cheat because it doesn’t have to, and it actually gives you mechanisms for checking that it is not cheating, e.g. entering dice rolls manually and a tutor mode that will correct your bad plays.
This phenomenon is repeated absolutely everywhere that user reviews of backgammon programs are available. On the Google Play store, I have seen this even on a program with an absolutely crappy algorithm that I can consistently beat on expert level as well as on decent ones that are backed by GNUbg. As a result, it is absolutely impossible to find out which are the good backgammon programs if you don’t already know, since they all get a low average rating.
We would all like to believe that… if we lose it is because of the good luck or cheating of our opponent.
When I’ve lost in the first round of single-elimination tournaments, I’ve found myself hoping that the person who beat me would prove skilled enough to win the entire tournament. That way, my loss wouldn’t mean that I totally sucked, but only that I wasn’t the best. So I think the quoted observation fails to account for nuances relating to how losses inform us about our skill level.
-- Matthew Leifer
I like the quote, but I have a nitpick:
When I’ve lost in the first round of single-elimination tournaments, I’ve found myself hoping that the person who beat me would prove skilled enough to win the entire tournament. That way, my loss wouldn’t mean that I totally sucked, but only that I wasn’t the best. So I think the quoted observation fails to account for nuances relating to how losses inform us about our skill level.