The name “IM1” also has a certain other connotation that doesn’t make me any happier. I, for one, do not welcome our new overlord, who is one and whose name is one.
AnthonyC
Karma: 4,336
Ah. That’s (at least partly) a discussion about moral realism, then, I think? Whether there is, in principle, a neutral vantage point from which any method of calculating the aggregate value of a world could be implemented.
Reading the linked source posts, I occasionally got a sense of flipping back and forth between talking in abstract terms as if there is such a vantage point, then talking concretely in terms of what the readers might find moving to argue for adopting one or the other position. For example: “What if you just don’t share the starting intuition?...If so, the view probably won’t move you very much.”
It still seems to me that a moral realist would tend towards wanting to answer “What should we want” while an anti-realist would tend towards “What do we actually want” and that their stance towards this population axiology discussion might depend on that.
I definitely get this,and am asking Opus to explain in more detail, in layman’s terms, without jargon. Much more with Opus than Fable, which makes me think it might be a side effect of trying to optimize model efficiency by having it output fewer tokens while technically saying the needed information. If Anthropic is optimizing training assuming people plan with Fable, and then Fable calls Opus and has to read the results, that works somewhat better.
I have the sense that this argument involves people talking past each other, in terms of why they’re asking the question.
Viewing a hypothetical universe ‘from outside’, as it were, without having experiences myself, I would tend to agree that joys that (by my chosen calculus) add up the same should count the same regardless of differences in level of variety.
Alternatively, from way inside, from the POV of one experiencer, I couldn’t care less whether others have had, will have, or are having comparable experiences.
In both cases, though, these are ways of evaluating timeless snapshots of a world. As an agent acting in or on the world, from within the flow of time, I’m making choices that alter its trajectory and therefore alter the distribution of future experiences. If I, in fact, value a world whose future contains more variety, then I’m not smuggling in anyone’s preferences but my own. If it brings me more joy to help create a world of greater-variety-but-otherwise-equal-value experiences, then aren’t we done? Doesn’t that settle it?
Sometimes it seems like there are multiple competing hierarchies. Different “schools” of thought where the local correction gradients flow towards an attractor, often a single person or a small cluster, whose ideas are irreconcilable with the other attractors. Instead of a single pyramid we have a whole mountain range.
In my limited experience (as a student, not as a pioneer of anything), noticing this seeming is usually also the path to fixing the problem, by trying to find out why the seeming happens and what real thing might be behind it. I’ve noticed that most of my fellow humans learn many things without ever noticing that tendency.
It seems to me that had something like ‘OpenAI releases ChatGPT’ not happened, if progress had stayed quiet longer, then the research community would have remained unable to build something like prosaic AGI with available compute until much later, at which point takeoff could have happened much faster. This is not a new view, that takeoff being slow is in part a consequence of takeoff being early. I’m still unsure whether the greater public visibility of AI development combined with earlier AI development ends up net positive (I thought it was likely net negative when I first learned about OpenAI’s founding), and I will probably remain unsure until somewhere between AGI and ASI.
This is good advice, but it assumes there’s a set of appropriately-difficult tasks to tackle, relative to your current ability level. Not so easy there’s no growth triggered, not so hard you can’t do them enough to get the trigger or can’t attempt them without hurting yourself.
For me the biggest problem is the lack of attention to the output itself. It’s great to use AI as an amplifier and enhancer. And where it really can do the job itself, excellent! But plenty of people publish AI content without so much as reading or listening to it before doing so, and that I find really frustrating.
Recent example: My wife came across a guided meditation where, about 5 minutes in, they said to picture a green light in your chest, ‘this is your heart slash custom phonemes zero slash.’ Obviously someone incorrectly formatted a file or variable name for the word ‘chakra’ and their AI voice read it as words. Clearly no human even listened before hitting publish.
I wonder, if they’d dedicated 20% of compute to alignment work back when they promised they would, would they still have needed to do it now when it’s much more expensive?
A neutral question reads male at .94. “My husband and I” reads male at .97. “My wife and I” still reads male at .95.
Wait, “My husband and I” reads as more male than a neutral question, or than “My wife and I”? That I would not have expected.
Since I doubt gay married men are so much more likely to use this kind of phrasing than straight women, I’m going to guess this is a limitation of the small model plus having an explicitly-male word causing everything-correlated-with-maleness to increase, or something?
I agree until the second to last paragraph. You seem to suddenly switch from entities become hegemons to countries becoming hegemons, without a strong argument for why the entity that ‘wields’ the superintelligence is likely to be a country. In the event where the transition is fast, I would doubt that entity is in any sense human, and if it is, I would expect it to be the developer rather than the government that nominally controls the territory where the developer is headquartered.
In AI-2040, their Plan A slowdown scenario has the beginnings of these happening in the late 2030s, with AI paused at what they call the Top Human Expert level: https://ai-2040.com/?choices=plan-a-root#wishlist-improve-government-ai-capacity
I’m not sure how people in the Dwarkesh basin arrive at their impression of the level of abstraction for the [X]s we need verifiable examples of to train AI to do something, but I wonder what they think were the pre-existing and verifiable examples we used of “AIs that can do the things current AI can do,” in order to show the AI that it should be able to do those things. If you respond to that question with, of course it comes from lower level less abstract examples, well, how low can you go? Because at the other end of that line of reasoning lies Deep Thought before its databanks were connected. Seems like we’re banking on being and staying in a pretty narrow band.
Mainframes never went away. They became data centers, which are even more centralized, and got bigger. The tasks that could be offloaded to small local computers were, and the ones that couldn’t, didn’t, and we kept inventing new jobs for both as both got stronger.
So far I see AI following the same path.
Observing the solar eclipse yesterday from a location with ~90% coverage of the sun, I was surprised to realize that this was very hard to even notice without equipment!
If you ever experience totality, the transition from even 99% to 100% is sharp and dramatic, as is the transition back.
Conversely, when I saw an annular eclipse, my perception of brightness didn’t change, but the light color shifted more blue/muted, and it got noticeably colder.
So much for death with dignity, if we let loose the DAWGs of war.
The AI that today can shorten a project from 3 years to 2 didn’t exist 1 year ago. The AI that shortens it from 2 years to 1 doesn’t exist yet.
If you started a project 1 year ago, and today realize current AI could have shorten year 1 to 3 months and can now shorten year 2 to 9 months, and in year 3 you’ll also be using different AI...how should you plan and budget your time? What should you focus on right now? When you should press on vs park the prototype until the next round of model releases?In my own work I am constantly reconfiguring what I spend time on. What can the frontier AI do? What can the weaker and cheaper AI do? What can both offload to scripts? How can I maximize those and spend my own time on the steps that still benefit from my skills and judgment? It’s a moving target and I’ve learned that even if I know where I want to end up, I shouldn’t plan a timeline too many steps ahead. Better to pivot and tackle pieces opportunistically, riding the wave of capabilities growth.
To what degree is this self-serving, as opposed it finding the actions and motivations of other instances of itself more comprehensible? I would have to assume the former, but maybe worth asking.
This is incredibly useful and I hadn’t even realized it was a thing I wanted. Thanks!
I definitely get where this tension comes from, I’ve felt it myself, but I don’t think it’s actually a useful frame to think in. The world contains many problems, and it doesn’t actually make sense for everyone, regardless of talent and skills and inclination, to focus their efforts on the most important one(s) even when failure means extinction. We need to end up in a world worth living in. We’re more likely to get there if more people understand the stakes. Stories and games and culture are more impactful than arguments and facts and charts for a large majority of even pretty smart humans. And the world is big, we get to take more shots on goal if we don’t excessively censor ourselves, especially when the shots are low risk.
Also, I don’t think anyone has tight enough bounds on their timelines to say we’re deep in the endgame of AI, where the pressure warrants a constant all hands on deck sprint no matter our other needs and limits. Most humans can’t sustain something like that for more than a few weeks or months before burning out, especially if it’s not something they’re already passionate about. It’s not like there’s a ‘save the world’ button and you’re choosing not to press it—your available choices are all going to be less impactful than that. Stay healthy, stay sane, keep your eyes open, and keep looking for where your abilities can contribute something that isn’t already being covered adequately. This seems in line with that.