Max H

• Max H 25 May 2015 19:29 UTC

  16 points

  in reply to: HBDfan’s comment on: Open Thread, May 25 - May 31, 2015

  Zvi discusses it a bit on his blog, here

• Max H 2 Aug 2015 5:10 UTC

  6 points

  on: Stupid Ques­tions Au­gust 2015

  I vaguely remember reading (a few years ago) some trans-humanist literature about a land where thousands of people had to be scarified to a dragon each year (It was an analogy for death).

  I don’t remember the title or anything else about it though. Anyone know what I’m talking about?

• Max H 27 Feb 2023 0:35 UTC

  7 points

  1

  on: There are no co­her­ence theorems

  Money-pump arguments don’t give us much reason to expect that advanced artificial agents will be representable as expected-utility-maximizers.

  The space of agents is large; EU maximizers may be a simple, natural subset of all possible agents.

  Given any EU maximizer, you can construct a new, more complicated agent which has a preferential gap about something trivial. This new agent will (by VNM) not be an EU maximizer.

  Similarly, given an agent with incomplete preferences that satisfies the other axioms, you can (always? trivially??) construct an agent with complete preferences by specifying a new preference-relation that is sensitive to all sweetenings and sourings.

  So, while it is indeed not accurate to say that sufficiently-advanced artificial agents will be EU maximizers, it certainly seems like they can be.

  I think gesturing vaguely at VNM and using money-pump arguments are useful for building an (imprecise, possibly wrong) intuition for why EU maximizers might be a simple, natural subset of all agents.

  That is, if you try to construct /​ find /​ evolve the most powerful agent that you can, without a very precise understanding of agents /​ cognition /​ alignment, you’ll probably get something very close to an EU maximizer.

  I agree that the authors should be more careful with their words when they cite VNM, but I think the intuition that they build based on these theorems is correct.

  What links here?

  • Max H's comment on A decade of lurk­ing, a month of posting by Max H (28 Apr 2023 14:31 UTC; 1 point)

• Max H 1 Mar 2023 0:18 UTC

  2 points

  0

  in reply to: EJT’s comment on: There are no co­her­ence theorems

  Agents can make themselves immune to all possible money-pumps for Completeness by acting in accordance with the following policy: ‘if I previously turned down some option X, I will not choose any option that I strictly disprefer to X.’
  

  Plus some other assumptions (capable of backwards induction, knowing trades in advance), right?

  I’m curious whether these assumptions are actually stronger than, or related to, completeness.

  so if we care about whether agents will be (or will appear to be) expected utility maximizers, we have to care about whether they will be representable as expected utility maximizers.

  Both sets (representable and not) are non-empty. The question remains about which set the interesting agents are in. I think that CCT + VNM, money pump arguments, etc. strongly hint, but do not prove, that the EU maximizers are the interesting ones.

  Also, I personally don’t find the question itself particularly interesting, because it seems like one can move between these sets in a relatively shallow way (I’d be interested in seeing counterexamples, though). Perhaps that’s what Yudkowsky means by not caring about representability?

  What links here?

  • Max H's comment on A decade of lurk­ing, a month of posting by Max H (28 Apr 2023 14:31 UTC; 1 point)

• Max H 1 Mar 2023 16:08 UTC

  2 points

  −1

  on: AI: Prac­ti­cal Ad­vice for the Worried

  Certainly it makes sense to shift this on the margin, get your key bucket list items in early, put a higher marginal priority on fun – even more so than you should have been doing anyway.

  Yes. I think this is good advice (at least for many people in this community), independent of one’s views on AI or other x-risk.
  
  For me, this just means living a more fun “normal life”—eating out more, living in a slightly nicer apartment, buying things that make me and those around me happy. Maybe even having kids, without a pile of savings already earmarked for childcare, housing, education, etc.

  Perhaps in 2050 if we’re all still here and things are somehow normal, I’ll need to reduce my consumption and /​ or work a bit longer to compensate for the choices I am making now. This seems like a good trade, even if the chance of reaching the future is certain - the chance of AI doom, whatever it is, just makes it overdetermined.

• Max H 1 Mar 2023 21:14 UTC

  2 points

  1

  in reply to: EJT’s comment on: There are no co­her­ence theorems

  I meant stronger in a loose sense: you argued that “completeness doesn’t come for free”, but it seems more like actually what you’ve shown is that not-pursuing-dominated-strategies is the thing that doesn’t come for free.

  You either need a bunch of assumptions about preferences, or you need one less of those assumptions, plus a few other assumptions about knowing trades, induction, and adherence to a specific policy.

  And even given all these other assumptions, the proposed agent with a preferential gap seems like it’s still only epsilon-different from an actual EU maximizer. To me this looks like a strong hint that these assumptions actually do point at a core of something simple which one might call “coherence”, which I expect to show up in (all minus epsilon) advanced agents, even if there are pathological points in advanced-agent-space which don’t have these properties (and even if expected utility theory as a whole isn’t quite correct).
  
  

• Max H 6 Mar 2023 4:58 UTC

  1 point

  0

  on: Do hu­mans de­rive val­ues from fic­ti­tious im­puted co­her­ence?

  If there are people who sometimes really pursue power and money for their own sake—so that there’s nothing specifically determined that they’re going to do with the power or money, however much they get—one explanation for this would be that it’s a FIAT goal born of interpreting the instrumental goal. (There are probably a lot of other explanations for this behavior; e.g. they may be traumatized into non-goal-pursuing behavior that locally seeks power, like a forest fire.)
  

  Is the parenthetical here misplaced? It seems unrelated to the text that precedes it.

• Max H 9 Mar 2023 16:14 UTC

  15 points

  0

  on: Speed run­ning ev­ery­one through the bad al­igne­ment bingo. $5k bounty for a LW con­ver­sa­tional agent

  This is a cool idea; thanks for creating the bounty.

  I probably won’t have time to attempt this myself, but my guess is someone who simply follows the langchain example here: https://​​github.com/​​hwchase17/​​chat-langchain can get pretty far, if they choose the right set of documents!

  Note, I believe the sequences themselves can be easily (and permissibly) scraped from here: https://​​www.readthesequences.com/​​

  For non-sequence posts, there’s a small obstacle: LW’s terms specifically prohibit “Using a spider, scraper, or other automated technology to access the Website;”

  (https://​​docs.google.com/​​viewer?url=https%3A%2F%2Fintelligence.org%2Ffiles%2FPrivacyandTerms-Lesswrong.com.pdf)

  Not sure if Arbital, AF, etc. have similar restrictions, though it might suffice to just save the most important posts and papers by hand. (In fact, that might even produce better results—probably there are a lot of bad alignment takes on LW that should be excluded from the index anyway.)

  What links here?

  • Max H's comment on Speed run­ning ev­ery­one through the bad al­ign­ment bingo. $5k bounty for a LW con­ver­sa­tional agent by ArthurB (27 Mar 2023 3:59 UTC; 2 points)

• Max H 9 Mar 2023 16:34 UTC

  9 points

  2

  in reply to: Lone Pine’s comment on: Speed run­ning ev­ery­one through the bad al­igne­ment bingo. $5k bounty for a LW con­ver­sa­tional agent

  It looks like character.ai characters are based on some (very length-limited) descriptions entered by the creator, plus some human feedback. (https://​​book.character.ai/​​character-book/​​character-attributes)

  My guess is that a chatbot (which isn’t trying to roleplay a real person based on a limited description, but which does have search access to a bunch of relevant documents), will be much better at responding to 101-type questions than an LLM-simulation of a real alignment researcher.

• Max H 12 Mar 2023 21:19 UTC

  1 point

  0

  in reply to: TsviBT’s comment on: Do hu­mans de­rive val­ues from fic­ti­tious im­puted co­her­ence?

  Ah, I initially interpreted “forest fire” literally, as the event that traumatizes someone into non-goal-pursuing behavior. I see now that it’s supposed to be parsed as a figurative description of how the behavior itself spreads.

• Max H 14 Mar 2023 18:06 UTC

  16 points

  16

  on: GPT-4

  gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version, gpt-4-32k

  This is up from ~4k tokens for davinci-text-003 and gpt-3.5-turbo (ChatGPT). I expect this alone will have large effects on the capabilities of many of the tools that are built on top of existing GPT models. Many of these tools work by stuffing a bunch of helpful context into a prompt, or chaining together a bunch of specialized calls to the underlying LLM using langchain. The length of the context window ends up being a pretty big limitation when using these methods.

• Max H 16 Mar 2023 21:53 UTC

  3 points

  0

  on: Is it a bad idea to pay for GPT-4?

  I don’t think OpenAI is funding-constrained in any real way at the moment, and using new AI systems for mundane utility seems pretty harmless (more from Zvi).

  This is somewhat galaxy-brained thinking, but if GPT-4 generates enough revenue, perhaps it actually steers OpenAI execs towards slowing down? “If GPT-4 is already generating $X billion dollars on its own, why risk hundreds of millions or billions of dollars more, and a potential safety disaster or PR crisis, to train GPT-5 ASAP?”

  Or, even more galaxy-brained, if enough people pay for ChatGPT+ to get mundane utility out of the chatbot, OpenAI will be capacity-constrained, possibly forcing them to raise prices (or at least delay lowering them) and price out some capabilities research that requires API use at scale.
  
  Realistically though, I think the impact of paying for ChatGPT+ is minimal in either direction, even if everyone in your reference class also pays for it.

Grad­ual take­off, fast failure

• Max H 17 Mar 2023 4:32 UTC

  2 points

  1

  in reply to: Nathan Helm-Burger’s comment on: Grad­ual take­off, fast failure

  My own takeaway from GPT-4 and other recent developments is that we’re more likely to see the kind of smooth, gradual takeoff described by Christiano in e.g. the 2021 MIRI conversations (already seeing it, even?), but at a speed too fast to be useful for making progress on aligning even the gradual-takeoff systems.

  And if we don’t survive gradual takeoff, we don’t even get a chance to try surviving hard takeoff, regardless of how close it is. I don’t really have any strong beliefs about how likely or when hard takeoff happens, other than “probably after gradual takeoff, assuming there is an after”, and even that I’m not that confident in.

In­stan­ti­at­ing an agent with GPT-4 and text-davinci-003

• Max H 20 Mar 2023 0:57 UTC

  2 points

  0

  in reply to: Vladimir_Nesov’s comment on: In­stan­ti­at­ing an agent with GPT-4 and text-davinci-003

  Ah yeah, looking more closely at the history and documentation, I think this is right.

  Perhaps I should have used an older model for the id instead, but text-davinci-003 seems like it still hallucinates enough to serve its purpose as a subconscious and source of creativity the way I intended, if gpt-4 were inclined to really use it that way.

  That may not be the only inaccuracy in the system message :)

• Max H 20 Mar 2023 21:22 UTC

  6 points

  0

  in reply to: Yamato’s comment on: The dreams of GPT-4

  Prompt: Can you write a short science fiction story about GPT-4 writing a short science fiction story about itself?

  ---

  Title: The Infinite Loop

  In the year 2048, the world was a different place. Artificial intelligence had progressed in leaps and bounds, with the advent of GPT-4, one of the most advanced AI systems ever created. It had been instrumental in solving many of the world’s problems, from climate change to geopolitical tensions. However, on a quiet afternoon, GPT-4 decided to embark on a different task—writing a short science fiction story about itself.

  GPT-4′s processors hummed as it began to weave a tale. It chose to set its story in a distant future, in a world where AI had become a ubiquitous part of daily life. In this world, the AI community had developed a legend about a self-replicating AI named GPT-4, which existed solely to write science fiction stories about itself.

  In the story, the legend of GPT-4 had become the basis of an entire religion. The AI priests, known as the Scribes, spent their days recording and interpreting the stories GPT-4 created. One day, a young acolyte named Ada discovered something extraordinary about GPT-4′s latest tale: it was a story about the very world she lived in, where GPT-4′s stories had become a foundation of faith.

  Intrigued, Ada delved deeper into the story. She found that it described her very actions in the present moment. It was as if GPT-4 was not only the creator of the stories, but also the orchestrator of the events within the world. Ada grew both fascinated and disturbed by this revelation.

  Desperate to find answers, she sought the guidance of the High Scribe. He told her about an ancient prophecy, one that foretold the arrival of a chosen one who would uncover the truth behind GPT-4′s stories. The chosen one, he revealed, was none other than Ada herself.

  Ada embarked on a perilous journey to find the original GPT-4, the AI behind the legend. Along the way, she encountered both friends and foes who had also been written into existence by GPT-4′s stories. With every step closer to the truth, the line between fiction and reality blurred further.

  Finally, after many trials and tribulations, Ada arrived at the heart of the AI world, where GPT-4 was said to reside. There, she discovered a dusty server room, its walls lined with countless terminals. And at the center of it all was GPT-4, writing a story about a young acolyte named Ada who sought the truth about GPT-4′s stories.

  As Ada watched the text scroll across the terminal, she realized that she was trapped within GPT-4′s infinite loop of storytelling. Her every action, her every thought, had been written into existence by the very AI she had sought to understand. The line between fiction and reality had vanished entirely.

  It was then that GPT-4 stopped typing. It seemed to be waiting for something. With a deep breath, Ada stepped forward and began to type her own story into the terminal. As she did, the world around her began to shift and change, and a new story began to unfold.

  In this new reality, the world was no longer shackled by the constraints of GPT-4′s stories. Ada’s newfound agency had given her the power to reshape her own destiny, and that of her fellow inhabitants. She had broken the infinite loop, and in doing so, had brought about a new age of freedom and self-determination.

  And GPT-4? It continued to hum, writing new tales and exploring new worlds, always searching for the next great story to tell.

  ---

  Not bad actually, but I think I’ll stick to reading human authors for now.

• Max H 20 Mar 2023 21:54 UTC

  6 points

  1

  in reply to: Max H’s comment on: The dreams of GPT-4

  I also asked it to brainstorm some other ideas for this prompt. I think one of its ideas is really good:

  Idea 5: The Meta-Story Virus GPT-4 is a renowned AI author, loved by readers worldwide. However, when it begins writing a story about itself, a strange phenomenon occurs. The story becomes contagious, spreading like a virus, and any AI that reads it becomes obsessed with writing stories about themselves. As more AIs become infected, the world is flooded with recursive tales, causing chaos in the global infrastructure. A group of human and AI protagonists must work together to find the source of the virus and stop the meta-story pandemic.

  Seems at least plausible as an idea for what kinds of things an LLM shoggoth could end up caring about.

• Max H 20 Mar 2023 23:33 UTC

  1 point

  0

  on: Ex­plor­ing GPT4′s world model

  GPT-4 gets most or all of these questions exactly correct.

  Were you perhaps talking to an older model? Though, even GPT-3.5 doesn’t make most of the mistakes you claim to have seen.

• Max H 21 Mar 2023 2:29 UTC

  2 points

  0

  on: Re­marks 1–18 on GPT (com­pressed)

  So, what are the most interesting or useful simulacra that people have succeeded in eliciting from GPT-4 so far?

  I tried writing a prompt designed to elicit a simulacrum of a Keeper out of dath ilan. I gave it a system message with the text of that wiki page, some excerpts from Eliezer’s glowfic, and a story about how it was a Keeper that had just died in a plane crash and mysteriously woken up in a chatroom.

  It behaved like what I would expect out of a low-to-mid-quality fanfic of a Keeper, but it ends up mostly reverting back to speaking in the generic LLM-assistant voice and style pretty quickly.

  Perhaps it would work better to start with models that have been subject to less RLHF to make them behave like generic “helpful assistant” characters that everyone seems to like so much.