DragonGod

“Hereti­cal Thoughts on AI” by Eli Dourado

[Question] Why The Fo­cus on Ex­pected Utility Max­imisers?

• DragonGod 8 Apr 2023 22:19 UTC

  LW: 90 AF: 26

  51

  AF

  on: GPTs are Pre­dic­tors, not Imitators

  GPTs are not Imitators, nor Simulators, but Predictors.

  I think an issue is that GPT is used to mean two things:

  1. A predictive model whose output is a probability distribution over token space given its prompt and context

  2. Any particular techniques/​strategies for sampling from the predictive model to generate responses/​completions for a given prompt.

  [See the Appendix]

  The latter kind of GPT, is what I think is rightly called a “Simulator”.

  From @janus’ Simulators (italicised by me):

  I use the generic term “simulator” to refer to models trained with predictive loss on a self-supervised dataset, invariant to architecture or data type (natural language, code, pixels, game states, etc). The outer objective of self-supervised learning is Bayes-optimal conditional inference over the prior of the training distribution, which I call the simulation objective, because a conditional model can be used to simulate rollouts which probabilistically obey its learned distribution by iteratively sampling from its posterior (predictions) and updating the condition (prompt). Analogously, a predictive model of physics can be used to compute rollouts of phenomena in simulation. A goal-directed agent which evolves according to physics can be simulated by the physics rule parameterized by an initial state, but the same rule could also propagate agents with different values, or non-agentic phenomena like rocks. This ontological distinction between simulator (rule) and simulacra (phenomena) applies directly to generative models like GPT.

  It is exactly because of the existence of GPT the predictive model, that sampling from GPT is considered simulation; I don’t think there’s any real tension in the ontology here.

  ---

  Appendix

  Credit for highlighting this distinction belongs to @Cleo Nardo:

  Remark 2: “GPT” is ambiguous

  We need to establish a clear conceptual distinction between two entities often referred to as “GPT” —

  • The autoregressive language model $\mu :T^k\to \Delta \left(T\right)$ which maps a prompt $x\in T^k$ to a distribution over tokens $\mu (\cdot |x)\in \Delta \left(T\right)$.

  • The dynamic system that emerges from stochastically generating tokens using $\mu$ while also deleting the start token

  Don’t conflate them! These two entities are distinct and must be treated as such. I’ve started calling the first entity “Static GPT” and the second entity “Dynamic GPT”, but I’m open to alternative naming suggestions. It is crucial to distinguish these two entities clearly in our minds because they differ in two significant ways: capabilities and safety.

  1. Capabilities:

     1. Static GPT has limited capabilities since it consists of a single forward pass through a neural network and is only capable of computing functions that are O(1). In contrast, Dynamic GPT is practically Turing-complete, making it capable of computing a vast range of functions.

  2. Safety:

     1. If mechanistic interpretability is successful, then it might soon render Static GPT entirely predictable, explainable, controllable, and interpretable. However, this would not automatically extend to Dynamic GPT. This is because Static GPT describes the time evolution of Dynamic GPT, but even simple rules can produce highly complex systems.

     2. In my opinion, Static GPT is unlikely to possess agency, but Dynamic GPT has a higher likelihood of being agentic. An upcoming article will elaborate further on this point.

  This remark is the most critical point in this article. While Static GPT and Dynamic GPT may seem similar, they are entirely different beasts.

  To summarise:

  • Static GPT: GPT as predictor

  • Dynamic GPT: GPT as simulator

  What links here?

  • The Com­pleat Cybornaut by ukc10014 (19 May 2023 8:44 UTC; 64 points)
  • Direc­tion of Fit by NicholasKees (2 Oct 2023 12:34 UTC; 32 points)

• DragonGod 13 Nov 2022 11:07 UTC

  69 points

  47

  on: Not­ing an un­sub­stan­ti­ated com­mu­nal be­lief about the FTX disaster

  I don’t buy this argument for a few reasons:

  • SBF met Will MacAskill in 2013 and it was following that discussion that SBF decided to earn to give

    • EA wasn’t a powerful or influential movement back in 2013, but quite a fringe cause.

  • SBF was in EA since his college days, long before his career in quantitative finance and later in crypto

  SBF didn’t latch onto EA after he acquired some measure of power or when EA was a force to be reckoned with, but pretty early on. He was in a sense “homegrown” within EA.

  The “SBF was a sociopath using EA to launder his reputation” is just motivated credulity IMO. There is little evidence in favour of it. It’s just something that sounds good to be true and absolves us of responsibility.

  Astrid’s hypothesis is very uncredible when you consider that she doesn’t seem to be aware of SBF’s history within EA. Like what’s the angle here? There’s nothing suggesting SBF planned to enter finance as a college student before MacAskill sold him on earning to give.

Sparks of Ar­tifi­cial Gen­eral In­tel­li­gence: Early ex­per­i­ments with GPT-4 | Microsoft Research

Tracr: Com­piled Trans­form­ers as a Lab­o­ra­tory for In­ter­pretabil­ity | Deep­Mind

[1911.08265] Mas­ter­ing Atari, Go, Chess and Shogi by Plan­ning with a Learned Model | Arxiv

Beren’s “De­con­fus­ing Direct vs Amor­tised Op­ti­mi­sa­tion”

The Limit of Lan­guage Models

Con­tra “Strong Co­her­ence”

[Question] Is In­struc­tGPT Fol­low­ing In­struc­tions in Other Lan­guages Sur­pris­ing?

“Dangers of AI and the End of Hu­man Civ­i­liza­tion” Yud­kowsky on Lex Fridman

[Yann Le­cun] A Path Towards Au­tonomous Ma­chine In­tel­li­gence

AI Risk Man­age­ment Frame­work | NIST

[Question] Is “Re­cur­sive Self-Im­prove­ment” Rele­vant in the Deep Learn­ing Paradigm?

• DragonGod 12 Feb 2023 10:30 UTC

  31 points

  22

  on: Ele­ments of Ra­tion­al­ist Discourse

  2. Non-Violence: Argument gets counter-argument. Argument does not get bullet. Argument does not get doxxing, death threats, or coercion.^[1]

  I’d want to include some kinds of social responses as unacceptable as well. Derision, mockery, acts to make the argument low status, ad hominems, etc.

  You can choose not to engage with bad arguments, but you shouldn’t engage by not addressing the arguments and instead trying to execute some social maneuver to discredit it.

[About Me] Cin­era’s Home Page

[Question] [Dis­cus­sion] How Broad is the Hu­man Cog­ni­tive Spec­trum?

I No Longer Believe In­tel­li­gence to be “Mag­i­cal”

Is Gen­eral In­tel­li­gence “Com­pact”?