I haven’t heard of any adversarial attacks, but I wouldn’t be surprised if they existed and were learnable. I’ve tried a variety of strategies, just for fun, and haven’t found anything that works except luck. I focused on various ways of forcing trades, and this often feels like it’s working but almost never does. As you can see, my record isn’t great.
It seems to be gaining a lot of ground by exploiting my poor openings. Maybe one strategy would be to memorise a specialised opening much deeper than usual? That could be enough. But it’d feel like cheating to me if I used an engine to find that opening. It’d also feel like cheating because it’s exploiting Leela’s lack of memory of past games. It’d be easy to modify it to deliberately play diverse games when playing against the same person.
Would you consider it cheating to observe a bunch of games between Leela and Stockfish, at every move predicting a probability distribution over what move you think Stockfish will play? That might give you an intuition for whether Leela is working by exploiting a few known blind spots (in which case you would generally make accurate predictions about what Stockfish would do, except for a few specific moves), or whether Leela is just out-executing you by a little bit per move (which would look like just being bad at predicting what Stockfish would do in the general case.
I don’t think that’d help a lot. I just looked back at several computer analyses, and the (stockfish) evaluation of the games all look like this:
This makes me think that Leela is pushing me into a complex position and then letting me blunder. I’d guess that looking at optimal moves in these complex positions would be good training, but probably wouldn’t have easy to learn patterns.
Oh, interesting! I didn’t expect to see a mix of games decided by many small blunders and games decided by a few big blunders.
I actually do suspect that there are learnable patterns in these complex positions, but I’m basing that off my experiences with a different game (hex, where my Elo is ~1800) where “the game is usually decided by a single blunder and recognizing blunder-prone situations is key to getting better” is perhaps more strongly true than of chess.
Yeah I didn’t expect that either, I expected earlier losses (although in retrospect that wouldn’t make sense, because stockfish is capable of recovering from bad starting positions if it’s up a queen).
Intuitively, over all the games I played, each loss felt different (except for the substantial fraction that were just silly blunders). I think if I learned to recognise blunders in the complex positions I would just become a better player in general, rather than just against LeelaQueenOdds.
I haven’t heard of any adversarial attacks, but I wouldn’t be surprised if they existed and were learnable. I’ve tried a variety of strategies, just for fun, and haven’t found anything that works except luck. I focused on various ways of forcing trades, and this often feels like it’s working but almost never does. As you can see, my record isn’t great.
I think I started playing it when I read simplegeometry’s comment you linked in your shortform.
It seems to be gaining a lot of ground by exploiting my poor openings. Maybe one strategy would be to memorise a specialised opening much deeper than usual? That could be enough. But it’d feel like cheating to me if I used an engine to find that opening. It’d also feel like cheating because it’s exploiting Leela’s lack of memory of past games. It’d be easy to modify it to deliberately play diverse games when playing against the same person.
Would you consider it cheating to observe a bunch of games between Leela and Stockfish, at every move predicting a probability distribution over what move you think Stockfish will play? That might give you an intuition for whether Leela is working by exploiting a few known blind spots (in which case you would generally make accurate predictions about what Stockfish would do, except for a few specific moves), or whether Leela is just out-executing you by a little bit per move (which would look like just being bad at predicting what Stockfish would do in the general case.
I don’t think that’d help a lot. I just looked back at several computer analyses, and the (stockfish) evaluation of the games all look like this:
This makes me think that Leela is pushing me into a complex position and then letting me blunder. I’d guess that looking at optimal moves in these complex positions would be good training, but probably wouldn’t have easy to learn patterns.
Oh, interesting! I didn’t expect to see a mix of games decided by many small blunders and games decided by a few big blunders.
I actually do suspect that there are learnable patterns in these complex positions, but I’m basing that off my experiences with a different game (hex, where my Elo is ~1800) where “the game is usually decided by a single blunder and recognizing blunder-prone situations is key to getting better” is perhaps more strongly true than of chess.
Yeah I didn’t expect that either, I expected earlier losses (although in retrospect that wouldn’t make sense, because stockfish is capable of recovering from bad starting positions if it’s up a queen).
Intuitively, over all the games I played, each loss felt different (except for the substantial fraction that were just silly blunders). I think if I learned to recognise blunders in the complex positions I would just become a better player in general, rather than just against LeelaQueenOdds.
Just tried hex, that’s fun.