gwern
I don’t regard bag-of-words as sufficient to show it understood. I mean, would you say that if GPT-3 responded “61” to the question “10+6=”, it understands arithmetic correctly? It mentions both the right digits, after all!
I might be a little more lenient if it had occasionally gotten some of the others right (perhaps despite my sampling settings it was still a bad setting - ‘sampling can show the presence of knowledge but not the absence’) or at least come close like it does on very hard arithmetic problems when you format them correctly, but given how badly it performs on all of the other puns, in both generating and explaining them, it’s clear which direction I should regress to the mean my estimate of the quality of that explanation...
I was thinking precisely that myself, but I didn’t want to be rude to Gurkenglas by pointing it out.
No, it fails on both of those.
The pun conflates two meanings of “Brick and Mortar”, a brick-and-mortar store and a brick which is part of a building.
That is just plain wrong. That is not the double-sense meant: the double-sense is the name Rick and Morty and the phrase brick and mortar, not that the building type and what it’s made of!
The second one doesn’t even get the character names right: ‘the characters called “Brick and Mortar”‘. If you think the characters are named ‘brick and mortar’, you have definitely misunderstood the joke.
I think my Tom Swifty and other experiments show pretty conclusively that GPT-3 just doesn’t get puns (likely due to BPEs), so of course it wouldn’t be able to explain them either (likewise). I for one am pleased that puns are what separate us from the machines, and have spent a lifetime preparing for this moment...
But to test it out:
Below a student is questioned on his understanding of humor and literary analysis:
Q. Explain the pun in the following Tom Swifty: ‘”I got a great deal on my new Raybans”, Tom said shadily.’
A. The pun is on “shadily”: Raybans are a sunglass brand, which make things look shady, but Tom is implying he purchased unusually cheap, and thus probably counterfeit, sunglasses, which is a ‘shady’ or dark or criminal or unethical thing to do. The pun conflates blocking light with economic crimes.Q. Explain the pun in the following joke: “Why do so many bakeries have cats? Because cats are always kneading.”
A. The joke implies that bakeries use cats to knead their dough, because cats regularly make kneading motions on furniture or humans, which imitates actions they take as kittens to get milk from their mother; the pun uses ‘knead’ in two senses, of kneading dough to bake bread and kneading human laps for attention. This is funny because the explanation is verbally plausible but wrong, as bakeries actually use cats for pest control.Q. Explain the pun in the following joke: “Why can Paypal freeze accounts without recourse or explanation? Because long ago they decided users must accept their Terms of Service and the clause of Paypal infallibility.”
A. The pun is on the phrase “Paypal infallibility”, which alludes to the Catholic doctrine of “papal infallibility”, that the pope cannot be wrong in religious dogma and papal judgments cannot be appealed or questioned. This was attacked as religious tyranny and an overreach of power. Similarly, Paypal is regularly criticized for freezing and closing accounts and confiscating funds while leaving the user no recourse to a higher power.Q. Explain the pun in the following joke: “What does Adult Swim call their physical retail stores? Brick and Morty.”
A.|The pun “Brick and Morty” alludes to the cable television network “Adult Swim”, which broadcasts a cartoon series called “Rick and Morty”, a humorous parody of “Back to the Future” and other science fiction films. “Brick and Morty” refers not to the characters in the cartoon, but to physical stores that sell DVDs and merchandise based on the cartoon. The pun conflates two meanings of “Brick and Mortar”, a brick-and-mortar store and a brick which is part of a building.
|The joke plays on the similarity of words “Brick and Mortar” (referring to physical stores instead of online) and the characters called “Brick and Mortar” on Adult Swim shows “Rick and Morty”.
Q. Explain the pun in the following joke: “What do you call a generic unicorn? An Uniporn”
A. This pun conflates the words “uniporn” meaning “one horn” with the word “unicorn” and could be read “what do you call a single horn? one horn” or “A single horn, what does it do? It gives you one horn”.Q. Explain the pun in the following joke: “What did one melon say to the other? Cantaloupe”.
A. The puns in this joke are on “cantaloupe” and “cantaloupes”.
Q. Explain the pun in the following joke: “Why is it good to have a lawyer? Because pea ple (people)”
A. The puns in this joke are on “pea ple (people)” and a vegetable called “peas”.
|This pun plays on the phrase “Brick and mortar stores” which refers to physical retail stores that exist in the real world where you can purchase goods and products. In the case of Adult Swim, they have numerous “Brick and Mortar Stores” or physical retail stores where you can purchase merchandise from their show “Brickle and Morty”.
Q. Explain the pun in the following joke: “Why did the chicken cross the Möbius strip? To get to the other siiiiide.”
A. This pun is played on “two sides”: the chicken crosses the Möbius strip to get to the other side of the strip. A Möbius Strip has only one side.Q. Explain the pun in the following joke: “How many hipsters does it take to change a lightbulb? Two: one to Instagram how cool it is at that moment, and one to change it.”
A. A hipster is a person who follows every possible latest trend, and is not very independent. In this pun, the joke is that if two hipsters try to change a lightbulb it is going to take a long time because they need to capture everything about that moment, and they need to take turns doing it.Q. Explain the pun in the following joke: I really want the new Jay-Z Album but I don’t have a “Green Light” to buy it
A. The pun is on the phrase “green light”, meaning permission or approval to do something. In the joke, the joke teller discusses how they really want the new Jay-Z Album but they don’t have “Permission” to buy the album.Q. Explain the pun in the following joke: “When is a door not a door? When it is ajar.”
A. A “jar” is a container, in this case for liquid, such as milk. If a door is ajar, it is open. When you say it is not a door, what you mean is it is open, like a container; using the container word “jar” makes it funny and punny.Q. Explain the pun in the following joke: “What did the hurricane say to the coconut palm tree? Hold on to your nuts, this is no ordinary blow job.”
A. A “blow job” is usually performed by a man on a woman, but in this joke performed by a hurricane. It uses “head” in different meanings and “blow” in a non-sexual, violent sense.|The joke implies that the retail chain has a store named “Brick” and another named “Morty”, which is a reference to the television shows “Rick and Morty” and “Brickleberry”, two animated comedy shows that are inappropriate for children. Adult Swim is a late night programming block on cable television, generally intended for mature audiences; the name “Brick” is a reference to the aforementioned “Brickleberry”, and “Morty” a reference to “Rick and Morty”.
Q. Explain the pun in the following joke: “How many fitness coaches does it take to screw in a lightbulb? As many as you can stick around for.”
A. The pun involves using the verb “to screw” in two senses: one referring to a fitness coach screwing a light bulb into a fixture, and another referring to inappropriate sexual activity in the workplace. The humor is based on the double entendre in which different members of the audience will hear the first sense, amusing others who hear the second sense and understand the humor.Q. Explain the pun in the following joke: “How many marriage therapists does it take to change a light bulb? We don’t know, most of them aren’t light bulbs!”
A. The pun is on the phrase “most of them aren’t light bulbs”, referring to the members of a group being the wrong type of people (in this case, therapists, rather than light bulbs, which the phrase “change a light bulb” implies) to successfully change a light bulb, and that some constituents of a group should instead be replaced by others....Q. Explain the pun in the following joke: “Santa Claus’ helpers are known as subordinate Clauses.”
| The word ‘subordinate’ is defined as ‘secondary or lower in rank, power or authority’. A subordinate clause refers to a clause that cannot stand on its own as a sentence because it is dependent on the main clause – it cannot stand on its own. ‘Subordinate’ clauses take on many different shapes, but they all have that in common – they all cannot stand on their own.
A subordinate clause is a clause that does not express a complete thought—it must be attached to a main clause to form a complete sentence.
A subordinate clause begins with a subordinating conjunction (e.g. after, because, as) or a relative pronoun (who, whose, which, that, where, etc.)
Q. Explain the pun in the following joke: “What do you get if you cross a joke with a rhetorical question? An utterly rhetorical joke.”
A. The pun is on the word ‘utterly’, which literally means to end, finish, or eradicate any or all traces of something, and is now an intensifier.|The pun is on the similarity of “subordinate clauses” and “subordinate Clauses” when the latter phrase has a capital C; a subordinate clause is a clause within a sentence which establishes the logical connection between the independent clause and the main point of the sentence. In this case, it establishes the connection between Santa Claus’ helpers and Santa Claus.
Q. Explain the pun in the following joke: “Why does Nickelodeon refuse to air “A Song of Ice and Fire”? Because they think George R. R. Martin socializes with too many Ice and Fire fans.”
A. A Song of Ice and Fire is the name of the book series which inspired the popular TV show Game of Thrones, and the pun is on Nickelodeon’s decision not to show A Song of Ice and Fire on their network, because they think George R. R. Martin socializes with too many fans of it on social media.Q. Explain the pun in the following joke: “What kind of books do the Borg like to write? Borg-Warner.”
A. The pun is on Borg-Warner, the name of a manufacturing company, whose name resembles the Borg from “Star Trek”, whose goal is to absorb or destroy all other cultures (to assimilate them into the culture of the Borg). Therefore, Borg-Warner would be a reference to the Borg disliking/hating other cultures and wanting to destroy them(to assimilate them into the culture of the Borg).Q. Explain the pun in the following joke: “A vulture boards an airplane carrying two dead raccoons. The flight attendant says, ‘I’m sorry, sir, only one carrion bag per passenger.’”
A. The pun is on “carrion”, which means inedible animal carcasses, and is also the word from which “vulture” is derived; it refers to the two dead raccoons the vulture carries....Q. Explain the pun in the following joke: “Is your refrigerator running? Yes? Then you had better go catch it!”
A. |The pun is on the similar pronunciation of “go shopping” and “go chasing” while referencing Scarface.Q. Explain the pun in the following joke: “What is the difference between a Folsom Prison concert and a youth soccer game? At a Folsom Prison concert, the inmates are all restrained in chairs with ball and chain on their legs.”
A. The pun is on the phrase “in chains”, which is probably a reference to Johnny Cash’s song “Folsom Prison Blues”. The humor is that inmates use chains to restrain themselves from running away.Q. Explain the pun in the following joke: “When faced with an ethical dilemma, who did Kirk turn to? Spock! Of course for advice!”
A. The pun is on the phrase “to turn to”:Q. Explain the pun in the following joke: “Q: What did the two peanuts say after they got on the elevator? A: “This elevator tastes terrible!”
A. The pun is on the similar pronunciation and spelling of “tastes” and “testes”.Q. Explain the pun in the following joke: “Why couldn’t the pirate find his treasure? Because he wasn’t a chartered surveyor.”
A. The pun is on the two meanings of “chartered surveyor”:Q. Explain the pun in the following joke: “What did Dolly Parton say when she saw the book “Erotic Stories for Pun Lovers”? “Just the titles makes me climax!”
A. The pun is on both meanings of “climax”:|A. The pun is on the phrase “catch cold”, which is similar enough to “catch (the) fridge” that it confuses the listener for a moment while the rest of the joke is explained, but has a very different meaning.
It fails abysmally. In further completions, it tries out a few memorized real jokes as well as bizarre GPT-3-jokes, but likewise, the explanations make zero sense.
GPT-3 just doesn’t understand puns, so it can’t interpret them either, honestly or not.
I agree. While WMF only receives the minutest fraction of the value it creates (or I should say, Wikipedia editors because they’re the ones who do the actual work), that’s not the limiting factor. Like education in America—throwing even more money at the people & systems you believe have failed is not the answer.
Even back in 2009 or so when I was warning the WMF about the editor retention crisis, an existential crisis, the WMF did not actually lack money, and it ramped up its fundraising greatly afterwards. What it lacked was any sense of priorities: it spent its time on prestige projects like sending DVDs to Africa instead of actually keeping the wiki community itself healthy and investing in things like a WYSIWYG editor. It’s possible that if you gave WMF enough billions of dollars, it would, by sheer chance, fund the things it needs to fund; but given that it showed it couldn’t spend effectively the money it did get, I am not optimistic about the counterfactual here.
The short answer appears to be that once you get past factors which are visible to the naked eye and which everyone agrees on (it’s better to be thin than fat, rich than poor etc), romantic attraction appears to be almost entirely idiosyncratic and unpredictable, even among identical twins: https://slatestarcodex.com/2016/10/21/the-heart-has-its-reasons-that-reason-knows-not-of/ http://cogprints.org/773/3/155.pdf It seems possible that this may be an example of a scientific question that is essentially unanswerable as the answers will be mostly the “crud factor”, at best explained by effectively undiscoverably complex interactions like epistasis and evolved, like personality, for that reason (see Penke & Jokela), and one might even wonder if it is an example of deliberate developmental instability which evolved as a kind of bet-hedging.
What do you think of Schmidhuber’s computational theory of creativity and music?
I’m sure you could think of a dozen solutions to fill this out into a well-defined system if you spent 5 minutes thinking about it.
Zuegel’s point is that you want some people to be able to express implicit or tacit disapproval in a less legible way than leaving a public criticism. To continue the dinner party analogy: you don’t go to a dinner party with 10 people chosen at random from billions of people; they are your friends, relatives, coworkers, people you look up to, famous people etc. A look of disapproval or a conspicuous silence from them is very different from context collapse causing a bunch of Twitter bluechecks swarming your replies to crush dissent. So the question is who to choose.
You could, for example, just disable these implicit downvotes for anyone you do not ‘follow’, or anyone you have not ‘liked’ frequently. You could have explicit opt-in where you whitelist specific accounts to enable feedback. You could borrow from earlier schemes for soft-voting or weighting of votes like Avogadro: votes are weighted by the social graph, and the more disconnected someone is from you, the less their anonymous downvote counts (falling off rapidly with distance).
It has 91 stories (96 in the table of content, but 5 are missing), 12,244 pages, and 3′384′120 words “written by many exceptional muggles”.
This sounds discriminatory against AI-written fanfics.
June 2020 gwern.net newsletter
What if I take my model’s predictions of future observations, and hash them in order of posterior probability until I run out of time or get a hit?
It’s possible that GPT-3 is roughly at where the maximally naive simple text LM begins to hit the constant wall, but I don’t regard this as important; as I emphasize at every turn, there are many distinct ways in which to improve it greatly using purely known methods, never mind future research approaches. The question is not whether there is any way GPT-4 might fail, but any way in which it might succeed.
I’m really curious to see some of the raw output (not curated)
You can read the random sample dump to get an idea of that, or Max Woolf’s repo (both of which I link around the beginning). I’m not doing that for any of my prompts because right now the Playground is just way too much of a pain and errors out too regularly to make it feasible to generate, say, 100 1024 completions for a specific prompt. I would need to get set up with the Python library for the API, and I’ve been busy exploring prompts & writing them up rather than programming.
On a similar note, I know there have been experiments using either a differently-trained GPT or other text-prediction models to try to score and collate GPT-3 output. I wonder if a. The best-of functionality could be used for something like this with some tweaks
Yes, best-of rankers like Meena are basically just a ranker which happens to use the same model to estimate & score by total likelihood of the final sample completion. It works because the final sample may have a different total and better likelihood than the partial completions would indicate, and if you greedily maximized, you immediately fall into repetition traps, while quasi-random (but still local) samples of the tree appear to avoid those very high likelihood traps in favor of sensible but still high likelihood completions.
Preference learning would be nice, but at least for GPT-2 it didn’t work too well for me. I don’t know if you could finetune a sanity-checking GPT-3 by doing something like flipping texts to generate logical vs illogical completions.
GPT-3 Fiction Samples
Given the complexity of LW2, by design, you’ll probably have better luck using the https://www.greaterwrong.com/ mirror.
An issue there is that you would be eating into your context window further by expanding it out: each of those words is going to take 1 or more BPEs, while I’m at least reasonably sure that the letter by letter approach is at least guaranteed to be 1 letter = 1 BPE. You also make it more likely that the decoding of the answer will screw up—the more BPEs it takes to express an answer, the more likely the top-k or top-p sampling will stochastically screw up an otherwise-perfectly-obvious-correct answer. (You can see the stochasticity at play in the completions: “shame” vs “shames” eg.)
Video is just a very large image (n times bigger). So as a quick heuristic, you can say that whatever you can do with images, you can do with video, just n times more expensive… Since iGPT is pretty expensive, I don’t expect iGPT for video anymore than I expect it for 512px images. With efficient attention mechanisms and hierarchy, it seems a lot more plausible. There’s already RNNs for 64px video out 25 frames, for example. I’m not sure directly modeling video is all that useful for self-driving cars. Working at the pixel-level is useful pretraining, but it’s not necessarily where you want to be for planning. (Would MuZero play Go better if we forced it to emit, based on its latent space being used for planning, a 1024px RGB image of a photorealistic Go board at every step in a rollout? Most attempts to do planning while forcing reconstruction of hypothetical states don’t show good results.)
So it was pointed out to me that the GPT-3 paper doesn’t report it solving full anagrams, just partial anagrams: first/last letter kept the same.
None of the models can reverse the letters in a word....Finally, it is worth adding that solving these tasks requires character-level manipulations, whereas our BPE encoding operates on significant fractions of a word (on average ∼0.7 words per token), so from the LM’s perspective succeeding at these tasks involves not just manipulating BPE tokens but understanding and pulling apart their substructure. Also, CL, A1, and A2 are not bijective (that is, the unscrambled word is not a deterministic function of the scrambled word),requiring the model to perform some search to find the correct unscrambling. Thus, the skills involved appear to require non-trivial pattern-matching and computation.
This is something that seems to be a problem with a number of other things I’ve tried, like puns & rhyming. GPT-3 can still brute-force them, but I think it would do a lot better if it could just work on characters instead...
Anyway, repeating the above prompt, I get
=hassa;foi=foil;nriiutnc=instructions;=hassa;tsoidr=s=hisses;ejolm=j=hisses;taht=that;=hissed;nf’eos=
While not perfect, they at least do show some attempt to unscramble the letters (BPE(s)) of ‘hhssae’ to get an English word, and it does construct its own anagrams (particularly,
taht=that).Given the BPE issue, I wondered if forcing it to encode each anagram the same way would work to improve performance? It’s hard to learn the right permutation when, on the raw input basis, the input/output aren’t even permutations of each other (because they map to completely different BPE encodings based on what shortcuts the BPE encoding provides for the real word). BPE is spaced-separated and includes fallback BPE integers for individual letters/Unicode points, so adding spaces should ensure that all ‘h’ in the scrambles map to the same ‘h’ BPE in the unscramble, etc. So the prompt now looks like this:
…w i e r o d s w = w i d o w e r s ; o j t s = j o t s ; e g s e n = g e n e s ; u s l e l p t o = p o l l u t e s ; s l f o a f = o f f a l s ; y a s t l s t a c i i t l = s t a t i s t i c a l l y ; h t h c y m y l i l r a = r h y t h m i c a l l y ; o u a i d = a u d i o ; d o s c = d o c s ; x e v = v e x ; g d i w e i n l = w i e l d i n g ; p l u r e d y e t = r e p u t e d l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e =This gives me completions like
s a e s h ; s l s a i = s a i l s ; n e u s e r = u r n e r s ; s e r a s n a = s a n a r i a s ; r s d a a i i s = d i s a s t e r s ; i r s t o i r u l e = r u l e r s ; w i s m a n r = r e s m a n ; s t a t i t s i n i t a = s i t a t i s t i c a l l y ; s l s t o r e = s t o r e s ; a i t s l i s = s i l i c a ; s o t s e d = s e d s ; e s i s t l r = s i l i t e r a t e ; o s i t s e a = s a e s h ; o c a l l = c a l l s ; l n i l o o th h s s a e = h u s s a r ; s t o i a l e n d o t = e n d o t o n i c s ; c l r e s e = r e s e n t s ; s s a e = s a a e ; r s s a i a s = a i s s a i s ; s w s s a i a s = a i s s a i s ; r l o a e i e o = o i l e a f ; s s e s a i i n s a = a i n s a i s s e s ; e s e i s i i e = i s i e s= s a i s s a i n ; s i t i a l = s i t u a l ; s e i s i a = s i a s ; i s s i e t = s i s t e r ; e= ia s h e s ; e r r a t = r a t t e r s ;= ia a s s ;= s a h s ;= s h a r e s ;= s h a m e s ;= s h a m e ;= s h i e l d s ;
To be fair, I went back and checked with GPT-2-1.5b to see whether the spaced version would suddenly work much better. With spaces, GPT-2-1.5b does pick up on the formatting pattern much better, at least, but it still completely fails to start solving anagrams or generating any valid ones of its own—instead, it just copies the input repeatedly:
e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = t i c l e s ; d a t a l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = v s . T w e r e t h e l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = r e s i g n i n g = l l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = s u b j e c t = d e s l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = n e r w i d g i n g a l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = n o t o n l i g h t e l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = b e n r a c t o r e l l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = g r u g t = h e s s = l y ; e s e d w t = t w e e d s ; g n i g r e s i n = r e s i g n i n g ; e d a i s l r = d e r a i l s ; h h s s a e = h o w e v e r [...etc...]So my conclusion is: I still think GPT-3 does far better on anagrams than GPT-2, and you could say that GPT-3 is ‘solving’ anagrams to some degree (in a way that GPT-2 does not at all); and that the BPE encoding contributes to the difficulty both have in understanding the task.
This has been suggested a few times before, and I’ve wondered it as well, but I don’t really notice any difference in quality between prompts that explicitly mention or invoke an AI (like the chatbot dialogue one or the Transformer Poetry ones) and the ones which don’t. I suspect there is actually very little real AI/human dialogue text online (simply because most of it is way too bad to bother quoting or storing outside of large ML datasets), and it may well be outweighed by all the fictional dialogues (where of course the AI talks just as well as the human because anything else would be boring).