Buck(Buck Shlegeris)

Karma: 5,686

Buck 19 Apr 2024 23:53 UTC
7 points
2
in reply to: Buck’s comment on: Express interest in an “FHI of the West”
I’ll also note that if you want to show up anywhere in the world and get good takes from people on the “how aliens might build AGI” question, Constellation might currently be the best bet (especially if you’re interested in decision-relevant questions about this).

Buck 19 Apr 2024 23:52 UTC
9 points
1
in reply to: owencb’s comment on: Express interest in an “FHI of the West”
(I work out of Constellation and am closely connected to the org in a bunch of ways)
I think you’re right that most people at Constellation aren’t going to seriously and carefully engage with the aliens-building-AGI question, but I think describing it as a difference in culture is missing the biggest factor leading to the difference: most of the people who work at Constellation are employed to do something other than the classic FHI activity of “self-directed research on any topic”, so obviously aren’t as inclined to engage deeply with it.
I think there also is a cultural difference, but my guess is that it’s smaller than the effect from difference in typical jobs.

Buck 19 Apr 2024 21:04 UTC
LW: 11 AF: 9
4
AF
on: Discriminating Behaviorally Identical Classifiers: a model problem for applying interpretability to scalable oversight
I like this post and this research direction, I agree with almost everything you say, and I think you’re doing an unusually good job of explaining why you think your work is useful.
A nitpick: I think you’re using the term “scalable oversight” in a nonstandard and confusing way.
You say that scalable oversight is a more general version of “given a good model and a bad model, determine which one is good.” I imagine that more general sense you wanted is something like: you can implement some metric that tells you how “good” a model is, which can be applied not only to distinguish good from bad models (by comparing their metric values) but also can hopefully be used to train the models.
I think that your definition of scalable oversight here is broader than people normally use. In particular, I usually think of scalable oversight as the problem of making it so that we’re better able to make a procedure that tell us how good a model’s actions are on a particular trajectory; I think of it as excluding the problem of determining whether a model’s behaviors would be bad on some other trajectory that we aren’t considering. (This is how I use the term here, how Ansh uses it here, and how I interpret the usage in Concrete Problems and in Measuring Progress on Scalable Oversight for Large Language Models.)
I think that it’s good to have a word for the problem of assessing model actions on particular trajectories, and I think it’s probably good to distinguish between problems associated with that assessment and other problems; scalable oversight is the current standard choice for that.
Using your usage, I think scalable oversight suffices to solve the whole safety problem. Your usage also doesn’t play nicely with the low-stakes/high-stakes decomposition.
I’d prefer that you phrased this all by saying:

It might be the case that we aren’t able to behaviorally determine whether our model is bad or not. This could be because of a failure of scalable oversight (that is, it’s currently doing actions that we can’t tell are good), or because of concerns about failures that we can’t solve by training (that is, we know that it isn’t taking bad actions now, but we’re worried that it might do so in the future, either because of distribution shift or rare failure). Let’s just talk about the special case where we want to distinguish between two models which and we don’t have examples where the two models behaviorally differ. We think that it is good to research strategies that allow us to distinguish models in this case.

Buck 17 Apr 2024 16:16 UTC
6 points
4
on: Staged release
without access to fine-tuning or powerful scaffolding.
Note that normally it’s the end user who decides whether they’re going to do scaffolding, not the lab. It’s probably feasible but somewhat challenging to prevent end users from doing powerful scaffolding (and I’m not even sure how you’d define that).

Buck 17 Apr 2024 15:59 UTC
11 points
8
in reply to: mesaoptimizer’s comment on: Paul Christiano named as US AI Safety Institute Head of AI Safety
I normally use “alignment research” to mean “research into making models be aligned, e.g. not performing much worse than they’re ‘able to’ and not purposefully trying to kill you”. By this definition, ARC is alignment research, METR and Redwood isn’t.
An important division between Redwood and METR is that we focus a lot more on developing/evaluating countermeasures.

Buck 17 Apr 2024 15:54 UTC
31 points
21
in reply to: habryka’s comment on: Paul Christiano named as US AI Safety Institute Head of AI Safety
Yeah I object to using the term “alignment research” to refer to research that investigates whether models can do particular things.
But all the terminology options here are somewhat fucked imo, I probably should have been more chill about you using the language you did, sorry.

Buck 16 Apr 2024 23:14 UTC
10 points
1
in reply to: habryka’s comment on: Paul Christiano named as US AI Safety Institute Head of AI Safety
I’m not saying you don’t need to do cutting-edge research, I’m just saying that it’s not what people usually call alignment research.

Buck 16 Apr 2024 23:00 UTC
35 points
27
in reply to: habryka’s comment on: Paul Christiano named as US AI Safety Institute Head of AI Safety
Why are you talking about alignment research? I don’t see any evidence that he’s planning to do any alignment research in this role, so it seems misleading to talk about NIST being a bad place to do it.

Buck 16 Apr 2024 19:27 UTC
7 points
3
in reply to: Buck’s comment on: nikola’s Shortform
Ugh I can’t believe I forgot about Rivest time locks, which are a better solution here.

Buck 16 Apr 2024 18:37 UTC
5 points
3
in reply to: nikola’s comment on: nikola’s Shortform
I feel pretty into encrypting the weights and throwing the encryption key into the ocean or something, where you think it’s very likely you’ll find it in the limits of technological progress

Buck 12 Apr 2024 0:31 UTC
8 points
3
on: Is LLM Translation Without Rosetta Stone possible?
Paul Christiano discusses this in “Unsupervised” translation as an (intent) alignment problem (2020)

Buck 4 Apr 2024 16:42 UTC
LW: 5 AF: 3
0
AF
in reply to: Fabien Roger’s comment on: Fabien’s Shortform
- If you are very good at cyber and extremely smart, you can hide vulnerabilities in 10k-lines programs in a way that less smart specialists will have trouble discovering even after days of examination—code generation/analysis is not really defense favored;
Do you have concrete examples?

Buck 2 Apr 2024 15:59 UTC
LW: 2 AF: 3
−7
AF
on: Sentience matters
The goal of alignment research is not to grow some sentient AIs, and then browbeat or constrain them into doing things we want them to do even as they’d rather be doing something else.
I think this is a confusing sentence, because by “the goal of alignment research” you mean something like “the goal I want alignment research to pursue” rather than “the goal that self-identified alignment researchers are pushing towards”.

Buck 30 Mar 2024 18:31 UTC
LW: 4 AF: 4
0
AF
in reply to: Richard_Kennaway’s comment on: Sentience matters
@So8res I’d be really interested in how you thought about these, especially the house elf example.

Buck 30 Mar 2024 17:04 UTC
LW: 63 AF: 29
38
AF
on: Catching AIs red-handed
A few months after posting this, I regret that we didn’t say something like the following:
It’s somewhat morally dubious to continue deploying AIs that are known to be scheming, because the fact that those AIs are scheming against us makes it more plausible that they’re unhappy about their situation. I’d like us to have thought about this more before using the plan described here.

Buck 20 Mar 2024 20:48 UTC
10 points
4
on: New report: Safety Cases for AI
I’m excited for this paper; I’ve often wanted to refer to a lot of these arguments/concepts and it’s great that there’s a carefully written source now.

Notes on control evaluations for safety cases

ryan_greenblatt, Buck and Fabien Roger

28 Feb 2024 16:15 UTC

32 points

0 comments32 min readLW link

Buck 6 Feb 2024 16:28 UTC
LW: 4 AF: 4
0
AF
in reply to: Seth Herd’s comment on: Toy models of AI control for concentrated catastrophe prevention
I agree re time-awareness, with two caveats:
- The kind of mechanism you listed probably only allows the AIs to have a rough idea of what time it is.
- We can keep some of the instances of our AI very unaware of the time, by restricting their between-episode memory. For example, we might do this for the instances responsible for untrusted monitoring, to reduce collusion.