I tried a different sort of prompt. The prompt ends, and GPT-3′s output begins, immediately after “MAGNUS:”.
This is an extract from a conversation between two top chess grandmasters, Magnus Carlsen and Hikaru Nakamura, who are talking about possible variants of the game of chess.
HIKARU: Suppose we made it so that instead of pawns being able to move only one or two squares on their first move, they can move up to six squares forward—assuming, of course, that the spaces they need to pass through are vacant. How do you think that would change the game?
MAGNUS: That would have a huge impact on the game. It would make it much easier for players to gain control of the center, as well as to launch more effective pawn storms. It could also lead to some interesting strategies, such as sacrificing a pawn to gain rapid development of pieces. Overall, I think it would be an interesting variant and could be quite a lot of fun to play.
I’m not sure what I think of “Magnus”’s response. It’s not clear to me that PGUS chess would make controlling the centre easier; it seems like it certainly would make pawn-storms easier to carry out; I’m not sure whether it would make it easier to sacrifice a pawn for faster piece development (PGUS would make it possible for the pawn to get more quickly out of the way of the pieces you want to develop, I guess? and maybe in the case where it gets taken immediately, whatever captures would be less likely to be in the way?) But at least one of the claims being made—the one about pawn storms—seems likely correct, important, and not 100% trivial.
A second response to the same prompt:
I think it would make the game a lot more dynamic, as pawns would be able to cover much more ground in their opening moves. It would also give them the opportunity to make more aggressive advances, which could lead to more interesting and complex strategic decisions. I think it could give the game a whole new level of excitement.
“A lot more dynamic” is somewhat plausible (though it’s in tension with the “easier to set up pawn chains” thing). The explanation for why feels a bit off—not wrong but too superficial. “More aggressive advances” is surely true but maybe trivial. “More interesting and complex strategic decisions” doesn’t really say much (and it seems perfectly possible to me that PGUS would make the game tactically more exciting but if anything simpler strategically, though you’d need to be a much stronger player than me before I’d believe your predictions on that front without playing lots of PGUS games). This one has a bit more of the “throw superficially plausible things at the wall and hope some of them sound good” feel that’s characteristic of a lot of GPT writing.
If a substantially smarter GPT-4 comes out in the near future, I think it’s likely that it will give more impressive answers to questions like these. I think I’d be on the other side of the first half of Omid’s bet from Omid. He’s probably right about the second half … except that I wouldn’t be astonished if it’s possible to make an AlphaZero-ish thing that, given a modestly different set of rules for a game it’s learned to play well, can do enough self-play fine-tuning training in say 20 minutes to play the modified game as well as a human who’s had 20 minutes to think about it. (It would have its code updated for the new rules, rather than being told them in prose. The latter would be a bigger challenge. I think “given 20 minutes to think about it” isn’t unreasonable, and using those 20 minutes to update its weights by self-play seems legit to me. I don’t know whether this is something one could do reasonably well now using, say, the existing LeelaChess codebase; my guess is that self-play learning is currently too slow.
I tried a different sort of prompt. The prompt ends, and GPT-3′s output begins, immediately after “MAGNUS:”.
I’m not sure what I think of “Magnus”’s response. It’s not clear to me that PGUS chess would make controlling the centre easier; it seems like it certainly would make pawn-storms easier to carry out; I’m not sure whether it would make it easier to sacrifice a pawn for faster piece development (PGUS would make it possible for the pawn to get more quickly out of the way of the pieces you want to develop, I guess? and maybe in the case where it gets taken immediately, whatever captures would be less likely to be in the way?) But at least one of the claims being made—the one about pawn storms—seems likely correct, important, and not 100% trivial.
A second response to the same prompt:
“A lot more dynamic” is somewhat plausible (though it’s in tension with the “easier to set up pawn chains” thing). The explanation for why feels a bit off—not wrong but too superficial. “More aggressive advances” is surely true but maybe trivial. “More interesting and complex strategic decisions” doesn’t really say much (and it seems perfectly possible to me that PGUS would make the game tactically more exciting but if anything simpler strategically, though you’d need to be a much stronger player than me before I’d believe your predictions on that front without playing lots of PGUS games). This one has a bit more of the “throw superficially plausible things at the wall and hope some of them sound good” feel that’s characteristic of a lot of GPT writing.
If a substantially smarter GPT-4 comes out in the near future, I think it’s likely that it will give more impressive answers to questions like these. I think I’d be on the other side of the first half of Omid’s bet from Omid. He’s probably right about the second half … except that I wouldn’t be astonished if it’s possible to make an AlphaZero-ish thing that, given a modestly different set of rules for a game it’s learned to play well, can do enough self-play fine-tuning training in say 20 minutes to play the modified game as well as a human who’s had 20 minutes to think about it. (It would have its code updated for the new rules, rather than being told them in prose. The latter would be a bigger challenge. I think “given 20 minutes to think about it” isn’t unreasonable, and using those 20 minutes to update its weights by self-play seems legit to me. I don’t know whether this is something one could do reasonably well now using, say, the existing LeelaChess codebase; my guess is that self-play learning is currently too slow.