An amusing story: I tried training a Connect4 bot purely to maximize entropy in a tournament system, so winning more games increased how often it played (the “selective pressure”)[1]. Naturally, I thought increasing the selective pressure would increase its game-winning abilities.
Nope! The stronger the selective pressure got, the longer it would drag out games. When it had a win in one move, it would mark it as one of the worst moves to play, only barely above immediately losing. It wouldn’t survive long enough to get entropy in the higher levels of the tournament, so it was trying to eke out more entropy by delaying winning as long as possible.
This is an oversimplification. It was closer to self-play with, “given a win/draw/loss and a tournament structure, how many more games would it get to play?”
An amusing story: I tried training a Connect4 bot purely to maximize entropy in a tournament system, so winning more games increased how often it played (the “selective pressure”) [1] . Naturally, I thought increasing the selective pressure would increase its game-winning abilities.
Nope! The stronger the selective pressure got, the longer it would drag out games. When it had a win in one move, it would mark it as one of the worst moves to play, only barely above immediately losing. It wouldn’t survive long enough to get entropy in the higher levels of the tournament, so it was trying to eke out more entropy by delaying winning as long as possible.
This is an oversimplification. It was closer to self-play with, “given a win/draw/loss and a tournament structure, how many more games would it get to play?”