Robert_AIZI

Com­par­ing An­thropic’s Dic­tionary Learn­ing to Ours

I was Wrong, Si­mu­la­tor The­ory is Real

The Tox­o­plasma of AGI Doom and Ca­pa­bil­ities?

Why do we as­sume there is a “real” shog­goth be­hind the LLM? Why not masks all the way down?

Re­search Re­port: Sparse Au­toen­coders find only 9/​180 board state fea­tures in OthelloGPT

My Reser­va­tions about Dis­cov­er­ing La­tent Knowl­edge (Burns, Ye, et al)

GPT-4: What we (I) know about it

How does GPT-3 spend its 175B pa­ram­e­ters?

• Robert_AIZI 14 Mar 2023 18:02 UTC

  34 points

  14

  on: GPT-4

  Gonna pull out one bit from the technical report, section 2.12:

  2.12 Acceleration
  OpenAI has been concerned with how development and deployment of state-of-the-art systems like GPT-4 could affect the broader AI research and development ecosystem.23 One concern of particular importance to OpenAI is the risk of racing dynamics leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines, each of which heighten societal risks associated with AI. We refer to these here as acceleration risk.”24 This was one of the reasons we spent eight months on safety research, risk assessment, and iteration prior to launching GPT-4. In order to specifically better understand acceleration risk from the deployment of GPT-4, we recruited expert forecasters25 to predict how tweaking various features of the GPT-4 deployment (e.g., timing, communication strategy, and method of commercialization) might affect (concrete indicators of) acceleration risk. Forecasters predicted several things would reduce acceleration, including delaying deployment of GPT-4 by a further six months and taking a quieter communications strategy around the GPT-4 deployment (as compared to the GPT-3 deployment). We also learned from recent deployments that the effectiveness of quiet communications strategy in mitigating acceleration risk can be limited, in particular when novel accessible capabilities are concerned.

  We also conducted an evaluation to measure GPT-4’s impact on international stability and to identify the structural factors that intensify AI acceleration. We found that GPT-4’s international impact is most likely to materialize through an increase in demand for competitor products in other countries. Our analysis identified a lengthy list of structural factors that can be accelerants, including government innovation policies, informal state alliances, tacit knowledge transfer between scientists, and existing formal export control agreements.

  Our approach to forecasting acceleration is still experimental and we are working on researching and developing more reliable acceleration estimates.

  My analysis:

  1. They’re very aware of arms races conceptually, and say they dislike arms races for all the right reasons (“One concern of particular importance to OpenAI is the risk of racing dynamics leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines, each of which heighten societal risks associated with AI.”)

  2. They considered two mitigations to race dynamics with respect to releasing GPT-4:

     1. “Quiet communications”, which they didn’t pursue because that didn’t work for ChatGPT (“We also learned from recent deployments that the effectiveness of quiet communications strategy in mitigating acceleration risk can be limited, in particular when novel accessible capabilities are concerned.”)

     2. “Delaying deployment of GPT-4 by a further six months” which they didn’t pursue because ???? [edit: I mean to say they don’t explain why this option wasn’t chosen, unlike the justification given for not pursuing the “quiet communications” strategy. If I had to guess it was reasoning like “well we already waited 8 months, waiting another 6 offers a small benefit, but the marginal returns to delaying are small.”]

  3. There’s a very obvious gap here between what they are saying they are concerned about in terms of accelerating potentially-dangerous AI capabilities, and what they are actually doing.

• Robert_AIZI 20 Nov 2023 15:28 UTC

  32 points

  9

  on: OpenAI Staff (in­clud­ing Sutskever) Threaten to Quit Un­less Board Resigns

  I’m noticing my confusion about the level of support here. Kara Swisher says that these are ⁵⁰⁵⁄₇₀₀ employees, but the OpenAI publication I’m most familiar with is the autointerpretability paper, and none (!) of the core research contributors to that paper signed this letter. Why is a large fraction of the company anti-board/​pro-Sam except for ⁰⁄₆ of this team (discounting Henk Tillman because he seems to work for Apple instead of OpenAI)? The only authors on that paper that signed the letter are Gabriel Goh and Ilya Sutskever. So is the alignment team unusually pro-board/​anti-Sam, or are the 505 just not that large a faction in the company?

  [Editing to add a link to the pdf of the letter, which is how I checked for who signed https://​​s3.documentcloud.org/​​documents/​​24172246/​​letter-to-the-openai-board-google-docs.pdf ]

No Really, At­ten­tion is ALL You Need—At­ten­tion can do feed­for­ward networks

In­vo­ca­tions: The Other Ca­pa­bil­ities Over­hang?

Is be­hav­ioral safety “solved” in non-ad­ver­sar­ial con­di­tions?

[Question] Ques­tion for Pre­dic­tion Mar­ket peo­ple: where is the money sup­posed to come from?

[Re­search Up­date] Sparse Au­toen­coder fea­tures are bimodal

Log-odds are bet­ter than Probabilities

Rat­ing my AI Predictions

• Robert_AIZI 20 Nov 2023 12:57 UTC

  21 points

  6

  on: Sam Alt­man, Greg Brock­man and oth­ers from OpenAI join Microsoft

  I hope this doesn’t lead to everyone sorting into capabilities (microsoft) vs safety (openai). OpenAI’s ownership was designed to preserve safety commitments against race dynamics, but microsoft has no such obligations, a bad track record (Sydney), and now the biggest name in AI. Those dynamics could lead to talent/​funding/​coverage going to capabilities unchecked by safety, which would increase my p(doom).

  Two caveats:

  • We don’t know what the Altman/​Brockman “advanced AI research team” will actually be doing at Microsoft, and how much independence they’ll have.

  • According to the new OpenAI CEO Emmett Shear, the split wasn’t due to “any specific disagreement on safety”, but I think that could be the end result.

Re­search Re­port: In­cor­rect­ness Cascades

• Robert_AIZI 14 Mar 2023 20:00 UTC

  18 points

  7

  in reply to: gwern’s comment on: GPT-4

  This non-news seems like it might be the biggest news in the announcement? OpenAI is saying “oops publishing everything was too open, its gonna be more of a black box now”.