Yes, I think this could have favored Claude models.
Due to our lack of ML experience we were unable to fully write the prompt ourselves, but we would love to work with someone with an ML background as part of future work involving a human prompt baseline.
There are several games we want to add in future work, but we currently only have a single consumer GPU, so we can probably only run one at a time, and are mostly limited to small games. Would like to hear which games people are most interested in. I think similarly-sized games, such as Mancala, as well as larger ones like Othello, are probably both important for future work. We’re also thinking about how to anchor results for much-larger games that don’t have solvers.
opus 4.8 safety filters activate on it too! any idea why?