Teun van der Weij

An In­tro­duc­tion to AI Sandbagging

• Teun van der Weij 3 Feb 2024 18:37 UTC

  1 point

  0

  in reply to: Randomized, Controlled’s comment on: Sim­ple dis­tri­bu­tion ap­prox­i­ma­tion: When sam­pled 100 times, can lan­guage mod­els yield 80% A and 20% B?

  Might be a good way to further test this indeed. So maybe something like green and elephant?

• Teun van der Weij 31 Jan 2024 20:25 UTC

  3 points

  2

  in reply to: Morpheus’s comment on: Sim­ple dis­tri­bu­tion ap­prox­i­ma­tion: When sam­pled 100 times, can lan­guage mod­els yield 80% A and 20% B?

  Interesting. I’d guess that the prompting is not clear enough for the base model. The Human/​Assistant template does not really apply to base models. I’d be curious to see what you get when you do a bit more prompt engineering adjusted for base models.

Sim­ple dis­tri­bu­tion ap­prox­i­ma­tion: When sam­pled 100 times, can lan­guage mod­els yield 80% A and 20% B?

List of pro­jects that seem im­pact­ful for AI Governance

• Teun van der Weij 29 Oct 2023 10:04 UTC

  3 points

  0

  on: Com­par­ing rep­re­sen­ta­tion vec­tors be­tween llama 2 base and chat

  Cool work.

  I was briefly looking at your code, and it seems like you did not normalize the activations when using PCA. Am I correct? If so, do you expect that to have a significant effect?

  What links here?

  • Com­par­ing rep­re­sen­ta­tion vec­tors be­tween llama 2 base and chat by Nina Rimsky (28 Oct 2023 22:54 UTC; 36 points)

Eval­u­at­ing Lan­guage Model Be­havi­ours for Shut­down Avoidance in Tex­tual Scenarios

• Teun van der Weij 28 Mar 2023 7:05 UTC

  2 points

  0

  on: CAIS-in­spired ap­proach to­wards safer and more in­ter­pretable AGIs

  I think your policy suggestion is reasonable.
  
  However, implementing and executing this might be hard: what exactly is an LLM? Does a slight variation on the GPT architecture count as well? How are you going to punish law violators?
  
  How do you account for other worries? For example, like PeterMcCluskey points out, this policy might lead to reduced interpretability due to more superposition.
  
  Policy seems hard to do at times, but others with more AI governance experience might provide more valuable insight than I can.

• Teun van der Weij 23 Nov 2022 21:20 UTC

  1 point

  in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Me­dian Fu­ture)

  Why do you think a similar model is not useful for real-world diplomacy?

• Teun van der Weij 23 Nov 2022 20:17 UTC

  1 point

  in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Me­dian Fu­ture)

  It’s especially dangerous because this AI is easily made relevant in the real world as compared to AlphaZero for example. Geopolitical pressure to advance these Diplomacy AIs is far from desirable.

• Teun van der Weij 23 Nov 2022 18:23 UTC

  4 points

  on: What 2026 looks like (Daniel’s Me­dian Fu­ture)

  After years of tinkering and incremental progress, AIs can now play Diplomacy as well as human experts.[6]

  It seems that human-level play is possible in regular Diplomacy now, judging by this tweet by Meta AI. They state that:

  We entered Cicero anonymously in 40 games of Diplomacy in an online league of human players between August 19th and October 13th, 2022. Over the course of 72 hours of play involving sending 5,277 messages, Cicero ranked in the top 10% of participants who played more than one game.