I’m interested in a measure of chess-playing ability that doesn’t depend on human players, and while perfect play would be the ideal reference, as long as chess remains unsolved, the other end of the spectrum, the engine whose algorithm is “list all legal moves and uniformly at random pick one of them,” seems the natural choice.
I read that the formula for Elo rating E is scaled so that, with some assumptions of transitivity of winning odds, pvictory≈11+10ΔE/400, so it’s trivial to convert probability to Elo rating, and my question is roughly equivalent to “What is the probability of victory of random play against, say, Stockfish 17?” If the Elo is close to 0[1], that makes the probability around 10−9 (estimating Stockfish 17′s Elo to be 3600). Eyeballing the y-intercept of this plot of lc0′s Elo vs. number of games of self-play, it looks something like 150–300 (lots of uncertainty). Does that sound reasonable?
I understand the probability of victory against the best modern engines is probably too small to accurately measure directly, so you would have to construct weaker/noisier engines and step down in increments of a few hundred Elo. Has anyone done this?
[Question] Chess—“Elo” of random play?
I’m interested in a measure of chess-playing ability that doesn’t depend on human players, and while perfect play would be the ideal reference, as long as chess remains unsolved, the other end of the spectrum, the engine whose algorithm is “list all legal moves and uniformly at random pick one of them,” seems the natural choice.
I read that the formula for Elo rating E is scaled so that, with some assumptions of transitivity of winning odds, pvictory≈11+10ΔE/400, so it’s trivial to convert probability to Elo rating, and my question is roughly equivalent to “What is the probability of victory of random play against, say, Stockfish 17?” If the Elo is close to 0[1], that makes the probability around 10−9 (estimating Stockfish 17′s Elo to be 3600). Eyeballing the y-intercept of this plot of lc0′s Elo vs. number of games of self-play, it looks something like 150–300 (lots of uncertainty). Does that sound reasonable?
I understand the probability of victory against the best modern engines is probably too small to accurately measure directly, so you would have to construct weaker/noisier engines and step down in increments of a few hundred Elo. Has anyone done this?
I don’t see an a priori reason to expect it would be.