I don’t think Matt Levine is too technical. I have zero finance knowledge, I’m not that smart (probably below average LW contributer), and am able to understand everything he writes
Szeth
Karma: 61
I don’t really understand why Scott says “I am scott alexander, far more connected to the singularity than almost anyone in history, therefore that’s evidence this is a simulation”
(1) Is he implying he thinks most people are P-zombies? If so, he’s implying p-zombies are possible which would be a surprising belief
(2) If p-zombies are possible then why make anyone conscious at all? What’s the point of only simulating people close to the singularity(3) Then for me personally, I have ~0 leverage over anything. I certainly follow the news closely and am connected in that sense, but I’m a random white collar worker.
(4) Would this be reason to be more confident the singularity is happening within our lifetime?
Makes sense. Not sure I agree but thanks for the responses.
If killing everyone, or killing enough that civilization is heavily destabilized, is on the table then preventing humans from taking back power is as well. The main thing that would prevent this imo is an AI being incapable of self sufficiency. Do you not think we are particularly close to either level of capability?
apologies if you’ve addressed this elsewhere
Not entirely related to this post, but I wonder how much of a crux takeover difficulty is. You talk a lot about how we don’t seem on track to have AIs capable of genuinely creative insights or anything close to sample efficiency with humans, and while I’m sympathetic to that take I don’t entirely understand your reasons for thinking it’s so important.
Even if we don’t have AIs close to sample efficiency with humans, they will still be able to self replicate and be given control over large swaths of our economy and be (in my view) capable of killing us all (via bioweapon most likely, but other possibilities exist). I doubt that they will take over unless they can be fully self sufficient, which I imagine to be a much higher bar than killing us all, but it still doesn’t seem to necessarily require truly novel creative insights (like cantors theorem) or sample efficiency.
Also (if by chance you are familiar with MTG) do you think we’ll get general-purpose AI systems which can beat humans in a new draft format? I suspect under your view of the world we won’t, though I’m not sure if this is particularly load bearing.
Is this true? I wasn’t on lesswrong back in the day but my I imagined if you told a random user that the two major AI labs would both be well aware of and trying to to mitigate the problem that would have been a positive update. And yes, profit incentives are stronger than perhaps would have been imagined, but that’s because AI progress is slow enough that they are able to be monetizable products something which is beneficial for our chances.
Thank you for the response.
Thank you
.
Thank you for the response. My compulsion is more like constantly rehashing the low p(doom) arguments in my head or even reopening the same LessWrong posts; eg I could probably recite Garriga-Alonso’s “Alignment Will Be Easy By Default” post word for word. Short term it offers some reprieve, but long term the main effect is just keeping me thinking about x-risk for longer.
This does differ from traditional OCD compulsions in a way I think works in my favor. When I leave my house I quadruple check the lock, convinced in some strange way that if I don’t check again the door will be unlocked and someone will rob me. The compulsion “works” insofar as checking relieves the anxiety because I believe, in some strange way, that it causally matters. Rehashing old arguments doesn’t have that property. It gives me temporary relief in a way that’s characteristic of a compulsion, but I don’t believe on any level that my thoughts causally affect our chances of making it out of this alive.
Because of that, I’ve started trying to just let the “we are all going to die” thought sit in my head without fighting it. Not repeating to myself that empirically alignment seems easier than anticipated, not rehearsing arguments about takeoff speeds. Just having the thought and not engaging. It’s been quite successful so far. The lock checking is a different story; I kinda can’t fight that impulse yet since checking the lock, at least in my mind, does have a causal effect on whether my house gets robbed. But I’m optimistic therapy can help there.
By “for worse” i mean that it is probably taken more seriously because AI capabilities have advanced so much, which is not something that is good imo
Thanks for the comment, though my name is Szeth because Seth was taken. I didn’t even enjoy The Way of Kings that much.
AI X risk is almost perfectly designed to oneshot people with OCD. The combination of absurdly high stakes, uncertainty, and a community dedicated to discussing it 24⁄7 with inquiry considered a virtue is tailor made to trigger one’s OCD. Further, it’s not exactly a subject that therapists would take seriously though, for better or for worse (and definitely for worse), I imagine that’s starting to change.
I had wondered for a while why I responded so much worse on a mental level than basically anyone else and, upon being diagnosed with Obsessive Compulsive Disorder, finally have an explanation. The typical treatment for OCD beyond medication is quite effective and called Exposure Response Therapy. With it, you practice not partaking in your compulsions and noticing that nothing bad happened. This sort of thing would work for someone obsessed with cleanliness but not someone who thinks there’s a 30% chance that we will all be dead in 10 years. That being said, my dealing with X risk is very characteristic of OCD in a way which makes me think I could respond well to therapy, however the treatment would likely, though not necessarily, be more bespoke than is typical.
In terms of how I was specifically affected: I would run and rerun the same anti-doom arguments in my head over and over, and was literally unable to stop thinking about it to the point of being dysfunctional. I’m better now, though not where I’d like to be.
I was wondering if there is anyone here who with OCD who would be willing to talk with me about their experience and what sorts of treatments they did to get better or put me in contact with someone who could. I would be willing to pay up to 1000$ for this, though given the unverifiable nature I’m not sure how I would go about this.
I just looked at the Google trends and it appears the term “existential threat” was very rare up until about 2009 and then steadily increases which does track well with OPs theory.
Interesting. Existential threat could make sense because AI is clearly a threat to the existence of SAAS companies and whatnot. Alignment is trickier to square and LW influence could definitely be the best explanation.
“Existential risk” here doesn’t necessarily come from lesswrong. Using the phrase “existential risk” to refer to your company going out of business makes perfect sense as it’s literally a risk to your companies existence. Alignment is a trickier one but even there the phrasing makes enough sense that it could plausibly not be lesswrong inspired.
I would put “Doesn’t know Zvi, played magic” at number two. I think you might be overestimating either how well known Zvi is or how likely one would be to know Zvi conditional on playing magic in the late nineties/early 2000s.
This doesn’t make a ton of sense to me. You think it’s more likely that JD knows Zvi but didn’t play magic than played magic but doesn’t know Zvi?
JD Vance very possibly knows who Zvi Mowshowitz is. Vance used to play the Magic the Gathering deck “Yawgmoth’s Bargain” which was largely designed by Zvi. Not sure what, if any, implications this has but certainly an interesting piece of trivia.
How “literally anything” are we talking about when you say “literally anything?” Could a superintelligence persuade me that tariffs are actually good? Possibly. Could a superintelligence persuade me that the 2008 Detroit Lions (who didn’t win a single game all year) are actually the best NFL team ever? I just don’t see it, but maybe I’m not being creative enough.