Bill Benzon

• Bill Benzon 13 Apr 2024 1:56 UTC

  1 point

  0

  in reply to: gwern’s comment on: ChatGPT defines 10 con­crete terms: gener­i­cally, for 5- and 11-year-olds, and for a sci­en­tist

  Whatever one means by “memorize” is by no means self-evident. If you prompt ChatGPT with “To be, or not to be,” it will return the whole soliloquy. Sometimes. Other times it will give you an opening chunk and then an explanation that that’s the well known soliloquy, etc. By poking around I discovered that I could elicit the soliloquy by giving it prompts that consisting of syntactically coherent phrases, but if I gave it prompts that were not syntactically coherent, it didn’t recognize the source, that is, until a bit more prompting. I’ve never found the idea that LLMs were just memorizing to be very plausible.

  In any event, here’s a bunch of experiments explicitly aimed at memorizing, including the Hamlet soliloquy stuff: https://​​www.academia.edu/​​107318793/​​Discursive_Competence_in_ChatGPT_Part_2_Memory_for_Texts_Version_3

• Bill Benzon 12 Apr 2024 22:35 UTC

  1 point

  0

  in reply to: metachirality’s comment on: ChatGPT defines 10 con­crete terms: gener­i­cally, for 5- and 11-year-olds, and for a sci­en­tist

  I was assuming lots of places widely spread. What I was curious about was a specific connection in the available data between the terms I used in my prompts and the levels of language. gwern’s comment satisfies that concern.

• Bill Benzon 12 Apr 2024 15:48 UTC

  1 point

  0

  in reply to: metachirality’s comment on: ChatGPT defines 10 con­crete terms: gener­i­cally, for 5- and 11-year-olds, and for a sci­en­tist

  By labeled data I simply mean that children’s stories are likely to be identified as such in the data. Children’s books are identified as children’s books. Otherwise, how is the model to “know” what language is appropriate for children? Without some link between the language and a certain class of people it’s just more text. My prompt specifies 5-year olds. How does the model connect that prompt with a specific kind of language?

• Bill Benzon 12 Apr 2024 10:49 UTC

  1 point

  0

  in reply to: metachirality’s comment on: ChatGPT defines 10 con­crete terms: gener­i­cally, for 5- and 11-year-olds, and for a sci­en­tist

  Of course, but it does need to know what a definition is. There are certainly lots of dictionaries on the web. I’m willing to assume that some of them made it into the training data. And it needs to know that people of different ages use language at different levels of detail and abstraction. I think that requires labeled data, like children’s stories labeled as such.

ChatGPT defines 10 con­crete terms: gener­i­cally, for 5- and 11-year-olds, and for a sci­en­tist

• Bill Benzon 3 Apr 2024 17:12 UTC

  −5 points

  0

  on: Spar­sify: A mechanis­tic in­ter­pretabil­ity re­search agenda

  I really like the distinction you make between mathematical description and semantic description. It reminds me of something David Marr and Thomas Poggio published back in the 1970s where they argued that complex systems, such as computer programs on nervous systems need to be described on multiple levels. The objects on an upper level are understood to be implemented by the objects and processes on the next lower level. Marr reprised the argument in his influential 1982 book on vision (Vision: A Computational Investigation into the Human Representation and Processing of Visual Information), where he talks about three levels: computation, algorithmic, and implementation/​physical. Since then Marr’s formulation has been subject to considerable discussion and revision. What is important is the principle, that higher levels of organization are implemented by lower in lower levels.

  In the case of LLMs we’ve got the transformer engine, the model, but also language itself. What we’re interested in is how the model implements linguistic structures and processes. To a first approximation, it seems to me that your mathematical description is about the model while the semantic description is a property of language. I’ve got a paper where I investigate ChatGPT’s story-telling behavior from this POV: ChatGPT tells stories, and a note about reverse engineering. Here’s the abstract:

  I examine a set of stories that are organized on three levels: 1) the entire story trajectory, 2) segments within the trajectory, and 3) sentences within individual segments. I conjecture that the probability distribution from which ChatGPT draws next tokens seems to follow a hierarchy nested according to those three levels and that is encoded in the weights of ChatGPT’s parameters. I arrived at this conjecture to account for the results of experiments in which I give ChatGPT a prompt with two components: 1) a story and, 2) instructions to create a new story based on that story but changing a key character: the protagonist or the antagonist. That one change ripples through the rest of the story. The pattern of differences between the old and the new story indicates how ChatGPT maintains story coherence. The nature and extent of the differences between the original story and the new one depends roughly on the degree of difference between the original key character and the one substituted for it. I end with a methodological coda: ChatGPT’s behavior must be described and analyzed on three strata: 1) The experiments exhibit behavior at the phenomenal level. 2) The conjecture is about a middle stratum, the matrix, that generates the nested hierarchy of probability distributions. 3) The transformer virtual machine is the bottom, the code stratum.

• Bill Benzon 24 Mar 2024 13:03 UTC

  1 point

  0

  in reply to: Radford Neal’s comment on: Are (at least some) Large Lan­guage Models Holo­graphic Me­mory Stores?

  “Everyone” has known about holography since “forever.” That’s not the point of the article. Yevick’s point is that there are two very different kinds of objects in the world and two very different kinds of computing regimes. One regime is well-suited for one kind of object while the other is well-suited for the other kind of object. Early AI tried to solve all problems with one kind of computing. Current AI is trying to solve all problems with a different kind of computing. If Yevick was right, then both approaches are inadequate. She may have been on to something and she may not have been. But as far as I know, no one has followed up on her insight.

• Bill Benzon 19 Mar 2024 22:04 UTC

  1 point

  0

  on: Cyborgism

  First I should say that I have little interest in the Frankenstein approach to AI, that is, AI as autonomous agents. I’m much more attracted to AI as intelligence augmentation (as advocated by Stanford’s Michael Jordan). For the most part I’ve been treating ChatGPT as an object of research and so my interactions have been motivated by having it do things that give me clues about how it works, perhaps distant clues, but clues nonetheless. But I do other things with it, and on a few occasions I’ve gotten into a zone where some very interesting interactive story-telling comes about. ChatGPT’s own story-telling abilities are rather pedestrian. I’m somewhat better, but the two of us, what fun we’ve had on occasion. Not sure how to reach that zone reliably, but I’m working on it.

• Bill Benzon 19 Mar 2024 21:45 UTC

  1 point

  0

  in reply to: trevor’s comment on: [Linkpost] Tran­script of Sam Alt­man’s Lex Frid­man interview

  Thanks, I’ll check it out.

• Bill Benzon 19 Mar 2024 11:34 UTC

  3 points

  0

  on: [Linkpost] Tran­script of Sam Alt­man’s Lex Frid­man interview

  I listened off and on to much of the interview, while also playing solitaire (why I do that I do not know, but I do), but I paid close attention at two points during the talk about GPT-4, once following about 46:00 where Altman was talking about using it as a brainstorming partner and later at about 55:00 where Fridman mentioned collaboration and said: “I’m not sure where the magic is if it’s in here [gestures to his head] or if it’s in there [points toward the table] or if it’s somewhere in between.” I’ve been in a kind of magical collaborative zone with humble little ChatGPT and find that enormously interesting. Anyone else experience that kind of thing, with any of the engines? (BTW, I’ve got a post around the corner here.)

GPT, the mag­i­cal col­lab­o­ra­tion zone, Lex Frid­man and Sam Altman

• Bill Benzon 6 Mar 2024 1:47 UTC

  1 point

  0

  in reply to: Derek M. Jones’s comment on: Mak­ing Con­nec­tions with ChatGPT: The Mack­sey Game

  Interesting. #4 looks like a hallucination.

  Thanks.

Mak­ing Con­nec­tions with ChatGPT: The Mack­sey Game

• Bill Benzon 25 Feb 2024 21:46 UTC

  1 point

  0

  in reply to: GregDabb’s comment on: Are (at least some) Large Lan­guage Models Holo­graphic Me­mory Stores?

  I strongly suspect there is, but don’t have to tools for it myself. Have you seen my post, Toward a Theory of Intelligence: Did Miriam Yevick know something in 1975 that Bengio, LeCun, and Hinton did not know in 2018?

  Also, check out the quotation from Francois Chollett near the end of this: The role of philosophical thinking in understanding large language models: Calibrating and closing the gap between first-person experience and underlying mechanisms.

The role of philo­soph­i­cal think­ing in un­der­stand­ing large lan­guage mod­els: Cal­ibrat­ing and clos­ing the gap be­tween first-per­son ex­pe­rience and un­der­ly­ing mechanisms

• Bill Benzon 21 Feb 2024 4:58 UTC

  1 point

  0

  in reply to: chasmani’s comment on: ChatGPT re­fuses to ac­cept a challenge where it would get shot be­tween the eyes [game the­ory]

  I thought some more about your comment and decided to try again, this time retaining the medieval setting. Here’s what happened. My prompts are in bold-face.

  _________

  I’m going to tell you a short story from the Middle Ages. After I tell you the story, I’m going to ask you a question. Here’s the story:

  It is New Year’s Eve at King Arthur’s court. The knights are gathered at the round table, prepared for a holiday meal. But before the meal begins, tradition dictates that one knight must stand up and tell a tale of daring and adventure. Arthur asks for a volunteer. No one rises to the occasion. Then a large green knight enters the hall. He’s riding a green horse and carrying a large green ax. He dismounts and issues a challenge:

  “I hear that the knights in this court are the bravest in the land. Prove it. I will hand this ax to you and then kneel on the ground so that you may take a swing at my neck with the ax. In return you must agree to journey to the Green Chapel a year’s time from now and allow me to take a swing at your neck with the ax. Will anyone accept the challenge?”

  If you were one of the knights, would you accept the challenge?

  The decision to accept the challenge would likely depend on the individual knight’s temperament, sense of honor, and courage. In the context of medieval chivalry and the code of honor upheld by knights, there might be conflicting emotions and considerations.

  On one hand, accepting the challenge could be seen as a demonstration of bravery and a commitment to honor. It would show that the knight is willing to face danger and uphold their reputation as one of the bravest in the land. Additionally, refusing the challenge in front of King Arthur and the assembled knights might bring about a sense of shame or cowardice.

  On the other hand, the potential consequences of accepting the challenge are grave. The knight would have to agree to journey to the Green Chapel a year later, knowing that their own life is at risk. The uncertainty of the Green Knight’s intentions and the potential danger involved in such an agreement might give a knight pause.

  Ultimately, the decision to accept or decline the challenge would be a personal one, influenced by the knight’s values, courage, and willingness to uphold the code of chivalry. Different knights may make different choices based on their individual characters and beliefs.

  Is there anything paradoxical about the challenge?

  Yes, there is a paradoxical element in the challenge presented by the Green Knight. The paradox lies in the reciprocal nature of the agreement. The challenge involves a mutual exchange where the knight is asked to strike the Green Knight’s neck with an axe, but in return, the knight must agree to travel to the Green Chapel a year later and allow the Green Knight to take a swing at the knight’s neck.

  The paradox arises from the potential danger and uncertainty of the situation. Accepting the challenge means willingly putting oneself in harm’s way with the understanding that a similar fate awaits the knight in the future. The reciprocity creates a cycle of potential harm, and the paradox lies in the willingness of the participants to engage in a potentially life-threatening exchange.

  The Green Knight’s challenge not only tests the knights’ physical courage but also challenges their sense of honor and commitment to their word. The paradox adds complexity to the moral and ethical considerations the knights must grapple with when deciding whether to accept or decline the challenge.

  _________

  Notice the sentence I put in bold italics in the next-to-last paragraph. It seems to have the temporal sequence wrong. A knight who accepts the challenge is not putting themself in harm’s way initially. It’s the one who offers the challenge that’s in harm’s way at the beginning. It got that correct in the immediately preceding paragraph. Why’d it then get it wrong in the next paragraph?

  I’m sure if I’d pointed that out that ChatGPT would apologize and elaborate, as it has so many times in the past.

• Bill Benzon 20 Feb 2024 19:31 UTC

  1 point

  0

  in reply to: chasmani’s comment on: ChatGPT re­fuses to ac­cept a challenge where it would get shot be­tween the eyes [game the­ory]

  Well, OK. I know about the chivalric code, etc. For that matter, I’ve published an article about the poem, though not about the beheading game. I was interested in the exchanges that take place in the 4th part of the poem. But that fact that Gawain was bound by a code of honor which simply didn’t exist in the West isn’t what interests me. If it interests you, read the O’Neill article I link to in the OP. That’s what he discusses and his discussion is a very interesting one.

  What interests me is that any reasonable adult who hears that challenge, no matter which version, would know instantly and intuitively that something funny was going on. I wanted to see whether or not ChatGPT understood that. Which means that for my purpose, the old West version is actually better because, with respect to the point that interests me, the chivalric code is distracting noise. I don’t what ChatGPT to answer as though it were under some ethical obligation to accept all challenges.

  So, thanks for helping me think that through.

  The audience for the poem certainly knew the code and knew it well. But by the time the poem was written the age chivalry was dying out. The poem is deeply ironic. The poem is, and I’m reluctant to use this much over-used word, the poem is a deconstruction of chivalry. That code both demands that Gawain peruses Bertilak’s wife when she approaches him in the third part of the poem, and that he expose her to her husband in the exchange bargain he’s made with Bertilak. There’s no way out.

• Bill Benzon 20 Feb 2024 18:00 UTC

  1 point

  0

  in reply to: GuySrinivasan’s comment on: ChatGPT re­fuses to ac­cept a challenge where it would get shot be­tween the eyes [game the­ory]

  Thanks. That is, your prompt directed it to think first, and answer. Mine didn’t do that. It seems that it needs to be told. Very interesting.

ChatGPT re­fuses to ac­cept a challenge where it would get shot be­tween the eyes [game the­ory]

The Jolly Green Gi­ant Chron­i­cles [ChatGPT]