“Successful, careful AI lab” was one of Holden’s 4 key “plays” in his 2023 misaligned AI risk reduction playbook, where he said
Concerns and reservations. This is a tough one. AI labs can do ~unlimited amounts of harm, and it currently seems hard to get a reliable signal from a given lab’s leadership that it won’t. (Up until AI systems are actually existentially dangerous, there’s ~always an argument along the lines of “We need to move as fast as possible and prioritize fundraising success today, to stay relevant so we can do good later.”) If you’re helping an AI lab “stay in the race,” you had better have done a good job deciding how much you trust leadership, and I don’t see any failsafe way to do that.
That said, it doesn’t seem impossible to me to get this right-ish (e.g., I think today’s conventional wisdom about which major AI labs are “good actors” on a relative basis is neither uninformative (in the sense of rating all labs about the same) nor wildly off), and if you can, it seems like there is a lot of good that can be done by an AI lab.
I’m aware that many people think something like “Working at an AI lab = speeding up the development of transformative AI = definitely bad, regardless of potential benefits,” but I’ve never seen this take spelled out in what seems like a convincing way, especially since it’s pretty easy for a lab’s marginal impact on speeding up timelines to be small (see above).
I do recognize a sense in which helping an AI lab move forward with AI development amounts to “being part of the problem”: a world in which lots of people are taking this action seems worse than a world in which few-to-none are. But the latter seems off the table, not because of Molochian dynamics or other game-theoretic challenges, but because most of the people working to push forward AI simply don’t believe in and/or care about existential risk ~at all (and so their actions don’t seem responsive in any sense, including acausally, to how x-risk-concerned folks weigh the tradeoffs). As such, I think “I can’t slow down AI that much by staying out of this, and getting into it seems helpful on balance” is a prima facie plausible argument that has to be weighed on the merits of the case rather than dismissed with “That’s being part of the problem.”
I think helping out AI labs is the trickiest and highest-downside intervention on my list, but it seems quite plausibly quite good in many cases.
This seems meaningfully different than Vitalik’s inevitabilism, so there does seem to be a steelman of this take.
I do agree with Eli that in Sam’s case it was a very convenient stance for him to endorse, and that I’d generally predict Sam’s outlook to be whatever maximises acquisition of his influence over humanity’s future lightcone.
“Successful, careful AI lab” was one of Holden’s 4 key “plays” in his 2023 misaligned AI risk reduction playbook, where he said
This seems meaningfully different than Vitalik’s inevitabilism, so there does seem to be a steelman of this take.
I do agree with Eli that in Sam’s case it was a very convenient stance for him to endorse, and that I’d generally predict Sam’s outlook to be whatever maximises acquisition of his influence over humanity’s future lightcone.