Had it got it right, that would have probably meant that it memorized this specific, very common question. Memorising things isn’t that impressive and memorising one specific thing does not say anything about capabilties as a one line program could “memorize” this one sentence. This way, however, we can be sure that it thinks for itself, incorrectly in this case sure, but still.
GPT also has problems with the Linda problem for the same reason:
https://twitter.com/dggoldst/status/1598317411698089984
Do people in that thread understand how gpt getting eg the ball+bat question wrong is more impressive than it getting it right or should I elaborate?
Please elaborate.
Had it got it right, that would have probably meant that it memorized this specific, very common question. Memorising things isn’t that impressive and memorising one specific thing does not say anything about capabilties as a one line program could “memorize” this one sentence. This way, however, we can be sure that it thinks for itself, incorrectly in this case sure, but still.