Yeah, admittedly health is kind of a borderline case where it’s technically factual but in practice mostly operates as a standard value-claim because of low entanglement and high reason to care.
I basically agree with your claim that the heuristic is approximating (reason to care) + (low entanglement).
A thing I am confused about: what is the medium-to-long-term actual policy outcome you’re aiming for? And what is the hopeful outcome which that policy unlocks?
You say “implement international AI compute governance frameworks and controls sufficient for halting the development of any dangerous AI development activity, and streamlined functional processes for doing so”. The picture that brings to my mind is something like:
Track all compute centers large enough for very high-flop training runs
Put access controls in place for such high-flop runs
A prototypical “AI pause” policy in this vein would be something like “no new training runs larger than the previous largest run”.
Now, the obvious-to-me shortcoming of that approach is that algorithmic improvement is moving at least as fast as scaling, a fact which I doubt Eliezer or Nate have overlooked. Insofar as that algorithmic improvement is itself compute-dependent, it’s mostly dependent on small test runs rather than big training runs, so a pause-style policy would slow down the algorithmic component of AI progress basically not-at-all. So whatever your timelines look like, even a full pause on training runs larger than the current record should less than double our time.
… and that still makes implementation of a pause-style policy a very worthwhile thing for a lot of people to work on, but I’m somewhat confused that Eliezer and Nate specifically currently see that as their best option? Where is the hope here? What are they hoping happens with twice as much time, which would not happen with one times as much time? Or is there some other policy target (including e.g. “someone else figures out a better policy”) which would somehow buy a lot more time?
Should be fixed now, thanks for flagging.
Should be fixed now, thanks for the heads-up. The same problem also broke images on a bunch of my other old posts; please do leave comments if you find more which I haven’t fixed yet.
Yeah, that’s one of the main things which the “causal models as programs” thing is meant to capture, especially in conjunction with message passing and caching. The whole thing is still behaviorally one big model insofar as the cache is coherent, but the implementation is a bunch of little sparsely-interacting submodel-instances.
I think there might be a simple miscommunication here: in our title and report we use “talent needs” to refer to “job and funding opportunities that could use talent.” Importantly, we generally make a descriptive, not a normative, claim about the current job and funding opportunities.
I think the title of this post is actively misleading if that’s what you’re trying to convey. “Defining” a term to mean something specific thing, which does not match how lots of readers will interpret it (especially in the title!), will in general make your writing not communicate what your “definition” claims to be trying to communicate.
If the post is about job openings and grant opportunities, then it should say that at the top, rather than “talent needs”.
Personally, I consider model-based RL to be not RL at all. I claim that either one needs to consider model-based RL to be not RL at all, or one needs to accept such a broad definition of RL that the term is basically-useless (which I think is what porby is saying in response to this comment, i.e. “the category of RL is broad enough that it belonging to it does not constrain expectation much in the relevant way”).
This model does allow for that. :) We can use this model whenever our two agents agree predictively about some parts of the world X; it’s totally fine if our two agents learned their models from different sources and/or make different predictions about other parts of the world.
You are a scholar and a gentleman.
Way back in the halcyon days of 2005, a company called Cenqua had an April Fools’ Day announcement for a product called Commentator: an AI tool which would comment your code (with, um, adjustable settings for usefulness). I’m wondering if (1) anybody can find an archived version of the page (the original seems to be gone), and (2) if there’s now a clear market leader for that particular product niche, but for real.
The word for a drug that causes loss of memory is “amnestic”, not “amnesic”. The word “amnesic” is a variant spelling of “amnesiac”, which is the person who takes the drug.
Thanks! Fixed now.
Oh yeah, I guess that could be a learning effect. When reading it I assumed the lack of need for repeating the numbers was just because the drug was wearing off.
Another class of applications which we discussed at the retreat: person 1 takes the amnestic, person 2 shares private information with them, and then person 1 gives their reaction to the private information. Can be used e.g. for complex negotiations: maybe it is in our mutual best interest to make some deal, but in order for me to know that I’d need some information which you don’t want to share with me, so I take the drug, you share the information, and I record some verified record of myself saying “dear future self, you should in fact take this deal”.
… which is cool in theory but I would guess not of high immediate value in practice, which is why the post didn’t focus on it.
I would love to hear suggestions for other things I could try. If you have any, let me know in a comment!
Do you know what the drug was which did this?
I actually find this kind of thinking quite useful. I mean, the particular policies proposed are probably pareto-suboptimal, but there’s a sound method in which we first ask “what policies would buy a lot more time?”, allowing for pretty bad policies as a first pass, and then think through how to achieve the same subgoals in more palatable ways.