GPT-3, belief, and consistency

I’ve seen a few peo­ple de­bat­ing what GPT-3 un­der­stands, and how this com­pares to hu­man un­der­stand­ing. I think there’s an eas­ier and more fruit­ful ques­tion to con­sider: what does it be­lieve?

It seems like it doesn’t be­lieve any­thing, or al­ter­nately, it be­lieves ev­ery­thing. It’s a cat­e­gory er­ror, like ask­ing what a library be­lieves, or what the In­ter­net be­lieves. But let’s go with that metaphor for a bit, be­cause it seems in­ter­est­ing to think about.

For a library, con­tra­dic­tions don’t mat­ter. A library can con­tain two books by differ­ent au­thors say­ing op­po­site things, and that’s okay since they are just be­ing stored. Maybe it’s bet­ter to think of GPT-3 as a large, in­ter­est­ingly-or­ga­nized mem­ory than as an agent? But like hu­man mem­ory, it’s lossy, and can mix up stuff from differ­ent sources, some­times in cre­ative ways.

How does GPT-3 re­solve in­con­sis­tency? If the In­ter­net is very con­sis­tent about some­thing, like the words to Jab­ber­wocky, then GPT-3 will be con­sis­tent as well. If there were two differ­ent ver­sions of Jab­ber­wocky that started the same and di­verged at a cer­tain point and they were equally pop­u­lar in the cor­pus, then it would prob­a­bly choose be­tween them ran­domly, if you have ran­dom­iza­tion turned on at all.

Some­times, GPT-3 can choose be­tween be­liefs based on style. Sup­pose that grade-school sci­ence ma­te­rial is writ­ten in one style and flat-earth rants are writ­ten in a differ­ent style. It wouldn’t be sur­pris­ing that GPT-3 would ap­pear to have differ­ent be­liefs about the shape of the earth based on which style of work it’s com­plet­ing. Or, if it can rec­og­nize an au­thor’s style, it might seem to have differ­ent be­liefs based on which au­thor it’s pre­tend­ing to be.

If GPT-3 can play chess, it’s due to on­line con­sen­sus about how to play chess. If we had two differ­ent chess-like games us­ing similar no­ta­tion then it might get them con­fused, un­less the con­text could be used to dis­t­in­guish them.

If base-10 and base-8 ar­ith­metic were equally com­mon in the cor­pus then I don’t think it could do ar­ith­metic very well ei­ther, though again, maybe it can dis­t­in­guish them from con­text. But if it doesn’t know the con­text, it would just guess ran­domly.

Of course, con­tra­dic­tions are ev­ery­where. We com­part­men­tal­ize. None of us are logic robots that halt when we find a con­tra­dic­tion. How­ever, con­tra­dic­tions of­ten bother us and we try to iron them out. Wikipe­dia con­trib­u­tors try to re­solve their in­con­sis­ten­cies through re­search, de­bate, or giv­ing up and say­ing that there’s no con­sen­sus and doc­u­ment­ing the con­tro­versy.

If you con­sider a search en­g­ine and Wikipe­dia to­gether as an al­gorithm for an­swer­ing ques­tions, you wouldn’t ex­pect it to re­solve in­con­sis­tency by re­turn­ing one ver­sion of an ar­ti­cle 40% of the time and the other 60% of the time, or by serv­ing up differ­ent ver­sions of the same ar­ti­cle based on stylis­tic differ­ences in how you ask the ques­tion. You might have to re­solve in­con­sis­tency your­self, but with static doc­u­ments that have dis­tinct ti­tles and URL’s, it’s eas­ier to see what you have.

GPT-3′s ways of re­solv­ing in­con­sis­tency hap­pen to work pretty well for some kinds of art and en­ter­tain­ment, but they’re not what we ex­pect of a fac­tual refer­ence, or even of a con­sis­tent fic­tional world.

This sug­gests some pos­si­ble ar­eas of re­search. What are smarter ways to re­solve in­con­sis­tency and how can we get an ma­chine learn­ing to use them? Is there some way to use ma­chine learn­ing to no­tice in­con­sis­tency in Wikipe­dia?

In the mean­time, I would guess that for fac­tual use, we will need to re­solve in­con­sis­ten­cies our­selves and feed our ma­chines a rel­a­tively con­sis­tent cor­pus. Feed­ing Wikipe­dia ar­ti­cles to the ma­chine means that the most glar­ing in­con­sis­ten­cies have been ironed out in ad­vance, which is why GPT-3 can an­swer fac­tual ques­tions cor­rectly some­times.

But if your in­ter­est is in fic­tion or in mak­ing in­ter­est­ing forg­eries, maybe you don’t care about this?