Been thinking a lot about whether it’s possible to stop humanity from
developing AI.
I think the answer is almost definitely not.
If it’s going to happen anyway, it seems like it would be good for someone
other than Google to do it first.
I mean, it’s unclear how much this was strategic rationalization or strategic lying.
This was a very convenient stance for Sam to endorse, which allowed him to pull together a high-powered coalition, which ultimately catapulted him into enormous power.
I think it’s likely that if this attitude hadn’t worked for allying with Elon, Sam would have ended up with a different strategic outlook that would also have been effective for rallying an amazing team.
“Successful, careful AI lab” was one of Holden’s 4 key “plays” in his 2023 misaligned AI risk reduction playbook, where he said
Concerns and reservations. This is a tough one. AI labs can do ~unlimited amounts of harm, and it currently seems hard to get a reliable signal from a given lab’s leadership that it won’t. (Up until AI systems are actually existentially dangerous, there’s ~always an argument along the lines of “We need to move as fast as possible and prioritize fundraising success today, to stay relevant so we can do good later.”) If you’re helping an AI lab “stay in the race,” you had better have done a good job deciding how much you trust leadership, and I don’t see any failsafe way to do that.
That said, it doesn’t seem impossible to me to get this right-ish (e.g., I think today’s conventional wisdom about which major AI labs are “good actors” on a relative basis is neither uninformative (in the sense of rating all labs about the same) nor wildly off), and if you can, it seems like there is a lot of good that can be done by an AI lab.
I’m aware that many people think something like “Working at an AI lab = speeding up the development of transformative AI = definitely bad, regardless of potential benefits,” but I’ve never seen this take spelled out in what seems like a convincing way, especially since it’s pretty easy for a lab’s marginal impact on speeding up timelines to be small (see above).
I do recognize a sense in which helping an AI lab move forward with AI development amounts to “being part of the problem”: a world in which lots of people are taking this action seems worse than a world in which few-to-none are. But the latter seems off the table, not because of Molochian dynamics or other game-theoretic challenges, but because most of the people working to push forward AI simply don’t believe in and/or care about existential risk ~at all (and so their actions don’t seem responsive in any sense, including acausally, to how x-risk-concerned folks weigh the tradeoffs). As such, I think “I can’t slow down AI that much by staying out of this, and getting into it seems helpful on balance” is a prima facie plausible argument that has to be weighed on the merits of the case rather than dismissed with “That’s being part of the problem.”
I think helping out AI labs is the trickiest and highest-downside intervention on my list, but it seems quite plausibly quite good in many cases.
This seems meaningfully different than Vitalik’s inevitabilism, so there does seem to be a steelman of this take.
I do agree with Eli that in Sam’s case it was a very convenient stance for him to endorse, and that I’d generally predict Sam’s outlook to be whatever maximises acquisition of his influence over humanity’s future lightcone.
In 2015 Sam Altman wrote to Elon Musk:
This seems to me like another example of what Vitalik calls Inevitibilism in his essay against Galaxy Brain Arguments.
I mean, it’s unclear how much this was strategic rationalization or strategic lying.
This was a very convenient stance for Sam to endorse, which allowed him to pull together a high-powered coalition, which ultimately catapulted him into enormous power.
I think it’s likely that if this attitude hadn’t worked for allying with Elon, Sam would have ended up with a different strategic outlook that would also have been effective for rallying an amazing team.
“Successful, careful AI lab” was one of Holden’s 4 key “plays” in his 2023 misaligned AI risk reduction playbook, where he said
This seems meaningfully different than Vitalik’s inevitabilism, so there does seem to be a steelman of this take.
I do agree with Eli that in Sam’s case it was a very convenient stance for him to endorse, and that I’d generally predict Sam’s outlook to be whatever maximises acquisition of his influence over humanity’s future lightcone.
Here are some things I did after reading If Anyone Builds It, Everyone Dies:
tell my friends and family about the book
write a review on Amazon & Goodreads
review scam copies on Amazon with 1 star
Searching for IABED on amazon.com, you will find that after the real book, several LLM written scam copies show up.
the same is true for amazon.co.uk, .de, … (these are distinct as far as reviews go)
I looked at the sample text and if it made no sense and was LLM written (it always was), wrote a 1 star review (only on amazon.de)
I only rated on amazon.de, because my Amazon account is only eligible for that. You could do it for the other Amazon sites.
Usually my reviews are display on the next day
(MIRI is aware but “it seems like there isn’t that much we can do about it, unfortunately.”)
(I also tried reporting them; but I am not sure they are breaking any rules.)
request the book in libraries
I found a list of German public libraries, where one can easily request them to buy a book and went through that. You could do this for libraries in your region.
I did 15 requests. 4 libraries bought it. 2 said they won’t buy it.
I’d attend a march, if 1000 people also pledged to march in Germany.
I might contact my representatives (How to email your politician).
Overview of Eliezer Yudkowsky’s writing:
on LessWrong starting with The Sequences / Rationality A-Z (book)
Harry Potter and the Methods Of Rationality (book)
Planecrash / Project Lawful (long fictional story)
on Arbital (AI Alignment)
on Twitter/X (mostly retweeting)
on Facebook (mostly retweeting)
on Medium (2 posts)
on Tumblr (fiction / on writing)
e.g. Masculine Mongoose (short story)
on fanfiction.net (3 stories)
on yudkowsky.net (personal/fiction)
on Reddit (fiction / on writing)
also pseudonymously: Kindness to Kin