Would you mind if I rewrote this in a less “manic” tenor, keeping the content and mood largely the same, and reposted? I like this essay and think the core of what you’re suggesting is reasonable, for reasons both stated and unstated, but I would like to try to say it differently in a way I think it will be taken better.
lc
Karma: 8,590
What an actually pessimistic containment strategy looks like
A non-magical explanation of Jeffrey Epstein
You can just spontaneously call people you haven’t met in years
POC || GTFO culture as partial antidote to alignment wordcelism
Stop posting prompt injections on Twitter and calling it “misalignment”
The U.S. is becoming less stable
My simple AGI investment & insurance strategy
Announcing $5,000 bounty for (responsibly) ending malaria
Yes, AI research will be substantially curtailed if a lab causes a major disaster
There are certain claims here that are concretely bad, but they’re also mixed in confusingly with what seem like nonsense complaints that are just… the reality of people spending extended time with other people, like:
“My roommates didn’t buy me vegan food while I was sick”
“Someone gives a lot of compliments to me but I don’t think they’re being genuine”
“I feel ‘low-value’”
If someone is being defrauded, yeah that’s one thing, but I’d rather not litigate “Is Kat/Emerson an asshole” in the court of public opinion.
In defense of flailing, with foreword by Bill Burr
FWIW you taking off the Mr. Nice guy gloves has actually made me make different life decisions. I’m glad you tried it even if it doesn’t work.
Addendum: A non-magical explanation of Jeffrey Epstein
If he is looking at as many profiles as he can and swiping right a reasonable portion of the time, this should never happen. As a paying customer, a large portion of the women he swipes right on will see his profile. He can view hundreds of profiles a day.
So if you go even one month with zero preliminary matches, and you are neither being super selective or highly unusually ugly, you know you are doing something very wrong with your profile or your profile pictures.
This is extremely embarassing to admit, but it seems like an important anecdote and I’ll say it anyways: in 2022 I spent a sizable amount of money on a “paid dating app service”. What that means is, I got on a call with a dating advisor who asked me questions for a few hours, and took notes. They helped me pick out new clothing, arranged a photographer to take my profile photos (which they checked up on photofeeler), and helped me write and setup profiles on Hinge, OkCupid, Tinder, Bumble, and Coffee Meets Bagel, for which I also elected to get premium subscriptions.
Then the service had another worker use those apps on my behalf full-time, for around ~50 hours a week, swiping on anything that seemed reasonable and then arranging text communication so that I could set up dates. While I was using this service, I checked in regularly to make sure that this worker was doing their job, that they were viewing new profiles, and that the conversations seemed solid.
I didn’t get literally zero matches, but after six months, I had only gone on one date. My advisor was so embarassed by my lack of success that when I told them I had to cancel for obvious reasons, she gave me free extensions for about two and a half months until she finally couldn’t swing that anymore.
I don’t think I’m particularly ugly; I’m around 23 and I’ve been in three relationships, with women who were not model gorgeous but (I think) were fairly attractive, one of which I ended up dating for six years. But I tried the entire motley crew of apps under what seemed like unnaturally strong conditions, and it just didn’t work.
I upvoted your post because it seems relatively lucid and raises some important points, but would like to say that I’m in the middle of writing a pretty long, detailed explanation of why I agree with most of the gripes (e.g. AIs can’t use magic to mine coal/build nanobots) and yet the object-level conclusions here are still untrue. In practice, I seriously doubt we would have more than a year to live after the release of AGI with the long term planning and reasoning abilities of most accountants, even without FOOM. People here shouldn’t assume that, because Eliezer never posted a detailed analysis on LessWrong, everyone on the doomer train is starting from unreasonable premises regarding how robot building and research could function in practice.
You say
Eliezer sounds good whenever he’s talking about a topic that I don’t know anything about.
But then you go on to talk about a bunch of philosophy & decision theory questions that no one has actual “expertise” in, except the sort that comes from reading other people talk about the thing. I was hoping Eliezer had said something about say, carpentry that you disagreed with, because then the dispute would be much more obvious and concrete. As it stands I disagree with your reasoning on the sample of questions I scanned and so it seems to me like this is sufficient to explain the dispute.
Suppose you’re in middle school, and one day you learn that your teachers are planning a mandatory field trip, during which the entire grade will jump off of a skyscraper without a parachute. You approach a school administrator to talk to them about how dangerous that would be, and they say, “Don’t worry! We’ll all be wearing hard hats the entire time.”
Hearing that probably does not reassure you even a little bit, because hard hats alone would not nudge the probability of death below ~100%. It might actually make you more worried, because the fact that they have a prepared response means school administrators were aware of potential issues and then decided the hard hat solution was appropriate. It’s generally harder to argue someone out of believing in an incorrect solution to a problem, than into believing the problem exists in the first place.
This analogy overstates the obviousness of (and my personal confidence in) the risk, but to a lot of alignment researchers it’s an essentially accurate metaphor for how ineffective they think OpenAI’s current precautions will turn out in practice, even if making a doomsday AI feels like a more “understandable” mistake.
When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.
No, you don’t, because Ziz-style violence is completely ineffective at improving animal welfare. It’s dramatic and self-destructive and might express soundly their factional leanings, but that doesn’t make it accomplish the thing in question.
Further, none of the murders & attempted murders the gang has committed so far seem to be against factory farm workers, so I don’t understand this idea that Ziz is motivated by ambitions of political terrorism at all. Reading their posts it sounds more like Ziz misunderstood decision theory as saying “retaliate aggressively all the time” and started a cult around that.
I’ve been impressed lately by how, while the EA forum has become basically overrun with useless scandal discussion, LessWrong has stayed virtually unafflicted. I think I’m the only person who ever commented about the Bostrom fiasco (in a shortform), and I feel bad about that and won’t do suchwise again. We must preserve our garden of autistic truth seeking and alignmentposts.