Dan H

• Dan HOct 23, 2023, 8:53 PM

  8 points

  1

  on: Ma­chine Un­learn­ing Eval­u­a­tions as In­ter­pretabil­ity Benchmarks

  I agree that this is an important frontier (and am doing a big project on this).

AISN #24: Kiss­inger Urges US-China Co­op­er­a­tion on AI, China’s New AI Law, US Ex­port Con­trols, In­ter­na­tional In­sti­tu­tions, and Open Source AI

AISN #23: New OpenAI Models, News from An­thropic, and Rep­re­sen­ta­tion Engineering

AISN #22: The Land­scape of US AI Leg­is­la­tion - Hear­ings, Frame­works, Bills, and Laws

Un­cov­er­ing La­tent Hu­man Wel­lbe­ing in LLM Embeddings

MLSN: #10 Ad­ver­sar­ial At­tacks Against Lan­guage and Vi­sion Models, Im­prov­ing LLM Hon­esty, and Trac­ing the In­fluence of LLM Train­ing Data

AISN #21: Google Deep­Mind’s GPT-4 Com­peti­tor, Mili­tary In­vest­ments in Au­tonomous Drones, The UK AI Safety Sum­mit, and Case Stud­ies in AI Policy

• Dan HAug 30, 2023, 2:11 AM

  23 points

  10

  on: Bro­ken Bench­mark: MMLU

  Almost all datasets have label noise. Most 4-way multiple choice NLP datasets collected with MTurk have ~10% label noise, very roughly. My guess is MMLU has 1-2%. I’ve seen these sorts of label noise posts/​papers/​videos come out for pretty much every major dataset (CIFAR, ImageNet, etc.).

AISN #20: LLM Pro­lifer­a­tion, AI De­cep­tion, and Con­tin­u­ing Drivers of AI Capabilities

• Dan HAug 21, 2023, 4:53 PM

  3 points

  0

  in reply to: O O’s comment on: AI Fore­cast­ing: Two Years In

  The purpose of this is to test and forecast problem-solving ability, using examples that substantially lose informativeness in the presence of Python executable scripts. I think this restriction isn’t an ideological statement about what sort of alignment strategies we want.

• Dan HAug 21, 2023, 4:52 PM

  3 points

  0

  in reply to: O O’s comment on: AI Fore­cast­ing: Two Years In

  I think there’s a clear enough distinction between Transformers with and without tools. The human brain can also be viewed as a computational machine, but when exams say “no calculators,” they’re not banning mental calculation, rather specific tools.

• Dan HAug 21, 2023, 4:25 PM

  3 points

  1

  in reply to: jsteinhardt’s comment on: AI Fore­cast­ing: Two Years In

  It was specified in the beginning of 2022 in https://​​www.metaculus.com/​​questions/​​8840/​​ai-performance-on-math-dataset-before-2025/​​#comment-77113 In your metaculus question you may not have added that restriction. I think the question is much less interesting/​​informative if it does not have that restriction. The questions were designed assuming there’s no calculator access. It’s well-known many AIME problems are dramatically easier with a powerful calculator, since one could bash 1000 options and find the number that works for many problems. That’s no longer testing problem-solving ability; it tests the ability to set up a simple script so loses nearly all the signal. Separately, the human results we collected was with a no calculator restriction. AMC/​​AIME exams have a no calculator restriction. There are different maths competitions that allow calculators, but there are substantially fewer quality questions of that sort.

  I think MMLU+calculator is fine though since many of the exams from which MMLU draws allow calculators.

• Dan HAug 20, 2023, 4:47 PM

  2 points

  0

  in reply to: Lech Mazur’s comment on: AI Fore­cast­ing: Two Years In

  Usage of calculators and scripts are disqualifying on many competitive maths exams. Results obtained this way wouldn’t count (this was specified some years back). However, that is an interesting paper worth checking out.

Risks from AI Overview: Summary

• Dan HAug 17, 2023, 5:52 PM

  15 points

  7

  on: An­nounc­ing Fore­sight In­sti­tute’s AI Safety Grants Program

  1. Neurotechnology, brain computer interface, whole brain emulation, and “lo-fi” uploading approaches to produce human-aligned software intelligence

  Thank you for doing this.

AISN #19: US-China Com­pe­ti­tion on AI Chips, Mea­sur­ing Lan­guage Agent Devel­op­ments, Eco­nomic Anal­y­sis of Lan­guage Model Pro­pa­ganda, and White House AI Cy­ber Challenge

AISN #18: Challenges of Re­in­force­ment Learn­ing from Hu­man Feed­back, Microsoft’s Se­cu­rity Breach, and Con­cep­tual Re­search on AI Safety

• Dan HAug 7, 2023, 1:39 AM

  8 points

  1

  on: how 2 tell if ur in­put is out of dis­tri­bu­tion given only model weights

  There’s a literature on this topic. (paper list, lecture/​slides/​homework)

AISN #17: Au­to­mat­i­cally Cir­cum­vent­ing LLM Guardrails, the Fron­tier Model Fo­rum, and Se­nate Hear­ing on AI Oversight

AISN #16: White House Se­cures Vol­un­tary Com­mit­ments from Lead­ing AI Labs and Les­sons from Oppenheimer