Psychology professor at University of New Mexico. BA Columbia, PhD Stanford. Works on evolutionary psychology, Effective Altruism, AI alignment, X risk. Worked on neural networks, genetic algorithms, evolutionary robotics, & autonomous agents back in the 90s.
geoffreymiller
Karma: 775
[Question] Best short introductions to AI safety & alignment for bright college students?
Thanks to Mikhail Samin for writing one of the most persuasive and important articles I’ve read on LessWrong.
I think a lot of the dubious, skeptical, or hostile comments on this post reflect some profound cognitive dissonance.
Rationalists and EAs generally were very supportive of OpenAI at first, and 80,000 Hours encouraged people to work there; then OpenAI betrayed our trust and violated most of the safety commitments that they made. So, we were fooled once.
Then, Rationalists and EAs were generally very supportive of Anthropic, and 80,000 Hours encouraged well-meaning people to work there; then Anthropic betrayed our trust and violated most of the safety commitments they made. So, we were fooled twice. Which is embarrassing, and we find ways to cope with our embarrassment, gullibility, and naivete.
What’s the lesson from OpenAI and Anthropic betraying our trust so massively and recklessly?
The lesson is simply about human nature. People are willing to sell their souls. A mid-level hit man is willing to kill someone for about $50,000. A cyber-scammer is willing to defraud thousands of elderly people for a few million dollars. Sam Altman was willing to betray AI Safety to achieve a current net worth of (allegedly) about $2.1 billion. Dario Amodei was willing to betray AI Safety to achieve his current net worth of (allegedly) about $3.7 billion. If the AI bubble doesn’t burst soon, they’ll each probably be worth over $10 billion within a couple of years.
So, we should have expected that almost anyone, no matter how well-meaning and principled, would eventually succumb to the greed, hubris, and thrills of trying to build Artificial Superintelligence. We like to think that we’d never sell our souls or compromise our principles for $10 billion. But millions of humans compromise their principles, every day, for much, much less than that.
Why exactly did we think Sam Altman or Dario Amodei would be any different? Because they were ‘friendlies’? Allies to the Rationalist cause? EA-adjacent? Long-termists who cared about the future?
None of that matters to ordinary humans when they’re facing the prospect of winning billions of dollars—and all they have to do is a bit of rationalization and self-deception, get some social validation from naive worshippers/employees, and tap into that inner streak of sociopathy that is latent in most of us.
In other words, Anthropic’s utter betrayal of Rationalists, and EAs, and humanity, should have been one of the least surprising developments in the entire tech industry. Instead, here we are, trading various copes and excuses for this company’s rapid descent from ‘probably well-intentioned’ to ‘shamelessly evil’.
Matrice—maybe, if it was possible for people to boycott Google/Deepmind, or Microsoft/OpenAI.
But as a practical matter, we can’t expect hundreds of millions of people to suddenly switch from gmail to some email alternative, or to switch from Windows to Linux.
It’s virtually impossible to organize a successful boycott of all the Big Tech companies that have oligarchic control of people’s digital lives, and that are involved in AGI/ASI development.
I still think the key point of leverage of specific, personalized, grassroots social stigmatization of AGI/ASI developers and people closely involved in what they’re doing.
(But I could be convinced that Big Tech boycotts might be a useful auxiliary strategy).
PS the full video of my 15-minute talk was just posted today on the NatCon YouTube channel; here’s the link
Matrice—for more on the stigmatization strategy, see my EA Forum post from a couple years ago, here
IMHO, a grassroots moral stigmatization campaign by everyone who knows AGI devs would be much more effective that just current users of a company’s products boycotting that company.
Oliver—that’s all very reasonable, and I largely agree.
I’ve got no problem with people developing narrow, domain-specific AI such as self-driving cars, or smarter matchmaking apps, or suchlike.
I wish there were better terms that could split the AI industry into ‘those focused on safe, narrow, non-agentic AI’ versus ‘those trying to build a Sand God’. It’s only the latter who need to be highly stigmatized.
Peace out :)
Seth—thanks for sharing that link; I hadn’t seen it, and I’ll read it.
I agree that we should avoid making AI safety either liberal-coded or conservative-coded.
But, we should not hesitate to use different messaging, emphasis, talking points, and verbal styles when addressing liberal or conservative audiences. That’s just good persuasion strategy, and it can be done with epistemic and ethical integrity.
Russell—I take your point that in most alternative timelines, we would already be dead, decades ago, due to nuclear war, and I often make that point in discussing AI risk, to hammer home the point that humanity does not have any magical ‘character armor’ that will protect us from extinction, and that nobody is coming to save us if we’re dumb enough to develop AGI/ASI.
However, I disagree with the claim that ‘our current situation is not one to preserve’. I know people in the military/intelligence communities who work full time on nuclear safety, nuclear non-proliferation, counter-terrorism, etc. There are tens of thousands of smart people across dozens of agencies across many countries who spend their entire lives reducing the risks of nuclear war. They’re not just activists making noise from outside the centers of power. They’re inside the government, with high security clearances, respected expertise, and real influence. I’m not saying the risk of nuclear war has gone to zero, but it is taken very seriously by all the major world governments.
By contrast, AI safety remains something of a fringe issue, with virtually no representation inside governments, corporations, media, academia, or any other power centers. That’s the thing that needs to change.
We don’t need a ‘hail Mary’ where we develop AGI/ASI and then hope that it can reduce nuclear risk more than it increases all other risks.
I didn’t say that all Rationalists are evil. I do consider myself a Rationalist in many ways, and I’ve been an active member of LessWrong and EA for years, and have taught several college courses on EA that include Rationalist readings.
What I did say, in relation to my claim that ‘they’ve created in a trendy millenarian cult that expects ASIs will fill all their material, social, and spiritual needs’, is that ‘This is the common denominator among millions of tech bros, AI devs, VCs, Rationalists, and effective accelerationists’.
The ‘common denominator’ language implies overlap, not total agreement.
And I think there is substantial overlap among these communities—socially, financially, ethically, geographically.
Many Rationalists have been absolutely central to analyzing AI risks, advocating for AI safety, and fighting the good fight. But many others have gone to work for AI companies, often in ‘AI safety’ roles that do not actually slow down AI capabilities development. And many have become e/accs or transhumanists who see humanity as a disposable stepping-stone to something better.
Yes. And way too much ‘AI safety work’ boils down to ‘getting paid huge amounts by AI companies to do safety-washing & public relations, to kinda sorta help save humanity, but without upsetting my Bay Area roommates & friends & lovers who work on AI capabilities development’.
Ok, let’s say we get most of the 8 billion people in the world to ‘come to an accurate understanding of the risks associated with AI’, such as the high likelihood that ASI would cause human extinction.
Then, what should those people actually do with that knowledge?
Wait for the next election cycle to nudge their political representatives into supporting better AI safety regulations and treaties—despite the massive lobbying and campaign contributions by AI companies? Sure, that would be nice, and it would eventually help a little bit.
But it won’t actually stop AGI/ASI development fast enough or decisively enough to save humanity.
To do that, we need moral stigmatization, right now, of everyone associated with AGI/ASI development.
Note that I’m not calling for violence. Stigmatization isn’t violence. It’s leveraging human instincts for moral judgment and social ostracism to negate the status and prestige that would otherwise be awarded to people.
If AI devs are making fortunes endangering humanity, and we can’t negate their salaries or equity stakes, we can at least undercut the social status and moral prestige of the jobs that they’re doing. We do that by calling them our as reckless and evil. This could work very quickly, without having to wait for national regulations or global treaties.
Well my toddler pronounces it ‘pee doom’
Kaj—I think the key thing here is to try to avoid making AI safety a strongly partisan-coded issue (e.g. ‘it’s a Lefty thing’ or ‘its a Righty thing’) -- but to find persuasive arguments that appeal about equally strongly to people coming from different specific political and religious values.
So, for example, conservatives on average are more concerned about ‘the dignity of work’ and ‘the sanctity of marriage’, so I emphasized how ASI would undermine both (through mass unemployment and AI sexbots/waifus/deepfakes), because that’s how to get their attention and interest. Whereas liberals on average may be more concerned about ‘economic inequality’, so when speaking with them, it might be more effective to talk about how ASI could dramatically increase wealth differences between future AI trillionaires and ordinary unemployed people.
So it’s really about learning specific ways to appeal to different constituencies, given the values and concerns they already have—rather than making AI into a generally liberal or generally conservative cause. Hope that makes sense.
Kaj: well, as I argued here, regulation and treaties aren’t enough to stop reckless AI development. We need to morally stigmatize anyone associated with building AGI/ASI. That’s the main lever of social power that we have.
I see zero prospect for ‘technical AI safety work’ solving the problem of slowing down reckless AI development. And it’s often little more than safety-washing by AI companies to make it look like they take safety seriously—while they continue to push AGI capabilities development as hard as ever.
I think a large proportion of the Rationalist/EA/LessWrong community is very naive about this, and that we’re being played by bad actors in the AI industry.
PS I’ve just posted the full text of my NatCon talk on AI risk here, along with a lot of introductory context. It might help guide a more constructive discussion here, insofar as people can see what I actually said.
My talk on AI risks at the National Conservatism conference last week
habryka—regarding what ‘aggression’ is, I’m coming to this from the perspective of having taught courses on animal behavior and human evolution for 35 years.
When biological scientists speak of ‘aggression’, we are referring to actual physical violence, e.g. hitting, biting, dismembering, killing, eating, within or between species. We are not referring to vocalizations, or animal signals, or their modern digital equivalents.
When modern partisan humans refer to ‘aggression’ metaphorically, this collapses the distinction between speech and violence. Which is, of course, what censors want, in order to portray speech that they don’t like as if it’s aggravated assault. This has become a standard chant on the Left: ‘speech = violence’.
I strongly disagree with that framing, because it is almost always an excuse for censorship, deplatforming, and ostracizing of political rivals.
I think to maintain the epistemic norms of the Rationality community, we must be very careful not to equate ‘verbal signals we don’t like’ with ‘acts of aggression’.
dr_s: How many MAGA supporters have you actually talked with, about AI safety issues?
It sounds like you have a lot of views on what they may or may not believe. I’m not sure how well-calibrated your views are.
Do you have a decent sample size for making your generalizations based on real interactions with real people, or are your impressions based mostly on mainstream news portrayals of MAGA supporters?
Richard—it’s true that not many people in the AI safety community are MAGA supporters, and that not many MAGA supporters are in the AI safety community.
The question is, why? Many on the Left, especially those involved in tech, have the stereotype that MAGA supporters are simply too stupid to understand AI safety issues. As a result, they simply haven’t bothered to reach out to the Right—and they socially ostracize and exclude anyone who seems to be on the Right.
Would Anthropic be excited to hire an overt MAGA supporter to join their AI safety team—however smart, competent, and committed they were? I doubt it.
TsviBT—I can’t actually follow what you’re saying here. Could you please rephrase a little more directly and clearly? I’d like to understand your point. Thanks!