I am a volunteer organizer with PauseAI US, a pro forecaster, and some other things that are currently much less important.
The risk of human extinction from artificial intelligence is a near-term threat. Time is short, the chance of catastrophe is high, and anyone can take simple, practical actions right now to help prevent the worst outcomes. Contact your political representatives!
But I don’t know where else to jump in.
Consider volunteering with an AI governance advocacy community. We can do a lot better then marginal technical safety work (most of which is just milling and justification, rather than actually tackling fundamental hard problems in AI safety). A global AI treaty is feasible.
The org I volunteer with (>100 grassroots meetings with Congressional offices so far this year, with many more coming soon): https://www.pauseai-us.org/
A sampling of other orgs with essentially the same mission: https://pauseai.info/ https://controlai.org/ https://www.torchbearer.community/ https://microcommit.io/ https://humansincontrol.org/
I don’t think “applause light” is being used correctly here. Leo’s claim is not devoid of content, it’s just a claim that most people agree with.
https://x.com/blueblimpms/status/2086574031529247183 It turns out this allegedly happened in April, and Pangram says that the original blogpost is AI generated.
Waymo has a feature where you can pay a few bucks for the car to wait extra time, which you can send while the car is waiting for you. I’ve used it once when I slightly mistimed a ride hail from the airport. I think Uber and Lyft could implement this, but it’s definitely lowest friction to implement when there is no driver and the fee is just set by the company.
Superintelligence safety isn’t “far worse” in China. You can’t get worse than zero superintelligence safety, which is what we currently have in both the US and China.
That argument for not-P doesn’t account for observation O.
In general, in my all-things-considered forecast, I expect that there is a significant chance of P. I think the basis for P is pretty clear at this point given the empirical data, and I believe these earlier arguments for P were correct. P is also just a direct extrapolation from R, so this ultimately shouldn’t be surprising. We’ve known P for a long time now, frankly.
A subset of the broader community is in favor of P, but I think that this is bad of them and that P is evil. Join me in calling your political representatives about P.
Being empathetic and being nice aren’t always the same thing. Loving actions are sometimes adversarial. When I imagine myself in the same position, I would want a friend to confront me and demand to know what the hell I’m thinking.
“We are doomed and there is nothing to be done, so I might as well doom us harder” can be made psychologically unsafe. It’s plenty of reason to end a friendship. They should feel pressure and shame for doing such a taboo thing, not to punish them, but because the lack of hard pushback acts as tacit approval, as if we can all be friends at the end of the day, as if friendship is more important than the death of all things.
This doesn’t work if cognitive ability itself is the dangerous part. AI killing us via macroscale manufacturing is a plausible floor for the strangeness of doom, not the ceiling. In reality, if a superintelligence can communicate with anyone, that on its own is exceedingly dangerous. If it gains internet access, it already sees “checkmate in 4058,” so to speak. It would need to be aligned before that point, which probably means it would need to be aligned before it finally gets good at philosophy.
Thank god for slow takeoff.
So far!
I think they’ll release an underwhelming product that is neither superintelligent nor safe, mostly as a bid for relevance.
I don’t think Ilya is any better than Sam, Dario, Demis, or Elon. They’re all AI company CEOs, which is a morally compromising thing to be. They’re all trying to build superintelligence, which is a very evil thing to try to do.
I’ve been saying for some time (out loud to my friends, anyway) that deduction is just a thousand inductions in a trenchcoat. I take this to be true in an abstract epistemological sense, but perhaps more importantly in the sense of the physical operation of our brains. It’s obvious to me that the human brain doesn’t contain a formal logic module. I don’t see why we should expect AI to be any different (aside from “it’s in a computer, and I think I know what computers are”).
But if you imitate how a human with certain emotions would act you are not having those emotions, you are just acting.
Certainly, but it turns out that LLMs do appear to functionally have emotions. Whether they feel emotions is an open question, but they do seem to have them.
Do I get Bayers Points?
Relatedly, if someone you know is accused of acting poorly toward another person, the observation that they never act that way toward you may provide nearly zero Bayesian evidence in their favor.
A very common example:
Alice: “My boyfriend is controlling and abusive toward me, and disrespectful of other women.”
Bob: “I don’t believe that. He never acted that way toward me!”
Alice: ”...Yeah. Duh. You’re not a woman.”
Outside of the specific example, and I’m not sure if you meant it this way, but I would caution against using “the world as we know it might end soon” as a reason to make decisions that would be bad if the world somehow stayed mostly normal.
To the specific example, I would search (or ask an LLM) for expert sources on raccoon behavior and welfare, and run with their conclusions.
Not that I know of, and I would also like to see such a report. It matches the angle I took in my contributions to the Metaculus Labor Hub.
but like many movements, the passionate, smart people are attracted into unproductive directions because they stick to what’s memetic, avoid conflicting with allies...
I was genuinely confused for a moment while reading this, because this is a perfect description of the pro-industry, anti-pause arm of rationalism and EA. Everyone working for an AI company (or their proxies like Coefficient Giving), or trying to “align” LLMs, or eschewing protests, or refusing to speak to politicians about shutting down the unconscionable suicide race… they are getting attracted into unproductive directions, sticking to what is often memetic in this community, and avoiding conflicting with their allies (often the people who are actively causing the AI crisis). In many cases, this is due not to bad epistemics, but to bad character. A disturbing number of EAs and Rats as here described are not pro-human by humanity’s standards or my own.
I agree with your conclusion that it was good to try to bring attention to the issue, and that enabled a lot of the good that is being done today. But the baton has rightfully been passed. With due thanks to early movers, the future of AI safety now belongs first and foremost to humanity.
“ASI Platform Provider” is a deeply cursed phrase.
This really hits on an emotional level, but I ultimately take a different perspective:
I think solving the relevant safety problems for any AI systems that could disrupt the world at this scale will be extremely difficult, and that most surviving worlds are ones in which AI progress has been ~halted via a long-term globally coordinated pause. That means the only futures I should expect to be alive to experience are ones that are fairly “normal” in many respects, which means normal futures are the ones I should plan for.
How that cashes out for me:
I don’t have any major long-term ambitions, since I’m so focused on pausing AI, but my partner for instance is working on a mathematics education PhD and wants to help reform math education in the US. I think there’s a very good chance that that will still be a valuable thing for her to pursue, despite how long-term it is.