superads91

Karma: 57

[untitled post]

superads914 Feb 2022 19:03 UTC

14 points

17 comments1 min readLW link

superads91 13 Mar 2022 12:38 UTC
11 points
on: The Proof of Doom
You can’t get such proof. A) the future is complex and hard to predict, b) there are even many plausible scenarios (not much likely, but plausible) where it doesn’t happen.

The best argument we can come up with is that it would be reasonable to assume a high probability (say 70-90%) that it will happen through x and y, and that probability is definitely too high for it not to become the most important thing of today.

Imo the best route is: Alpha Code can already compete with human programmers in some areas. Will it take long before AI becomes better than humans at programming, and can therefore take care of AI development by itself and rewrite its own code?

I’ve found that people often don’t agree with AI x-risk because they haven’t been exposed to the key concepts like recursive self improvement and basic AI drives. I believe that once you understand those concepts it’s hard not to change your mind, because they paint such a clear picture.

superads91 7 Apr 2022 4:52 UTC
8 points
in reply to: JBlack’s comment on: MIRI announces new “Death With Dignity” strategy
“That said, I’m not convinced that permanent Holocaust is worse than permanent extinction, but that’s irrelevant to my point anyway.”

Maybe it’s not. What we guess are other people’s values is heavily influenced by our own values. And if you are not convinced that permanent Holocaust is worse than permanent extinction, then, no offense, but you have a very scary value system.

“If someone isn’t convinced by the risk of permanent extinction, are you likely to convince them by the (almost certainly smaller) risk of permanent Holocaust instead?”

Naturally, because the latter is orders of magnitude worse than the former. But again, if you don’t share this view, I can’t see myself convincing you.

And we also have no idea if it really is smaller. But even a small risk of an extremely bad outcome is reason for high alarm.

superads91 28 Feb 2022 18:44 UTC
8 points
in reply to: ztzuliios’s comment on: Russia has Invaded Ukraine
″ If NATO missile defense can change the calculus such that retaliation from a first strike seems survivable”

Lol. Survive retaliation? Depends on what you mean by surviving. Maybe only get 70% of your country destroyed instead of 100%, maybe only get 70% of the population subsequently die from nuclear winter? Not much of a survival.

Stuxnet the Russian arsenal? Are you serious? That barely worked in a baby nuclear arsenal, do you think it would work in the nation with the greatest nuclear arsenal, and with some of the most capable communities of cyber warfare?

Why would NATO want to pretty much “just almost” destroy the world just to invade Russia?

superads91 6 Apr 2022 18:03 UTC
7 points
in reply to: johnlawrenceaspden’s comment on: MIRI announces new “Death With Dignity” strategy
“I love violence and would hope that Mars is an utter bloodbath.”

The problem is that biological violence hurts like hell. Even most athletes live with chronic pain, imagine most warriors. Naturally we could solve the pain part, but then it wouldn’t be the violence I’m referring to. It would be videogame violence, which I’m ok with since it doesn’t cause pain or injury or death. But don’t worry, I still got the joke!

“”don’t kill anyone and don’t cause harm/suffering to anyone”

The problem with this one is that the AI’s optimal move is to cease to exist.”

I’ve thought about it as well. Big brain idea: perhaps the first AGI’s utility function would be to act in the real world as minimally as possible, maybe only with the goal of preventing other people from developing AGI, keeping like this until we solve alignment? Of course this latter part of policing the world would be already prone to a lot of ambiguity and sophism, but again, if we program do not’s (do not let anyone else build AGI, plus do not kill anyone, plus do not cause suffering, etc) instead of do’s, it could lead to a lot less ambiguity and sophism, by drastically curtailing the maneuver space. (Not that I’m saying that it would be easy.) As opposed to like “cure cancer”, or “build an Eiffel tower”.

“And that’s already relying on being able to say what ‘kill someone’ means in a sufficiently clear way that it will satisfy computer programmers”

I don’t think so. When the brain irreversibly stops you’re dead. It’s clear. This plays into my doubt that perhaps we keep underestimating the intelligence of a superintelligence. I think that even current AIs could be made to discern when a person is dead or alive, perhaps even better than us already.

“For instance, when Captain Kirk transports down to the Planet-of-Hats, did he just die when he was disassembled, and then get reborn? Do we need to know how the transporter works to say?”

Maybe don’t teletransport anyone until we’ve figured that out? There the problem is teletransportation itself, not AGI efficiently recognizing what is death at least as well as we do. (But I’d venture saying that it could even solve that philosophical problem, since it’s smarter than us.)

“Stuart Russell is a very clever man, and if his approach to finessing the alignment problem can be made to work then that’s the best news ever, go Stuart!

But I am a little sceptical because it does seem short on details, and the main worry is that before he can get anywhere, some fool is going to create an unaligned AI, and then we are all dead.”

I gotta admit that I completely agree.

“Whereas alignment looks harder and harder the more we learn about it.”

I’ll say that I’m not that much convinced about most of this that I’ve said. I’m still way more to the side that “control is super difficult and we’re all gonna die (or worse)”. But I keep thinking about these things, to see if maybe there’s a “way out”. Maybe we in this community have built a bias that “only the mega difficult value alignment will work”, when it could be false. Maybe it’s not just “clever hacks”, maybe there are simply more efficient and tractable ways to control advanced AI than the intractable value alignment. But again, I’m not even that much convinced myself.

“and I am pretty sure that the author of ‘Failed Utopia 4-2’ has at least considered the possibility that it might not be so bad if we only get it 99%-right.”

Exactly. Again, perhaps there are much more tractable ways than 100% alignment. OR, if we could at least solve worst-case AI safety (that is, prevent s-risk) it would already be a massive win.

superads91 13 Mar 2022 12:34 UTC
7 points
in reply to: lorepieri’s comment on: The Proof of Doom
“Overall I don’t see a clear argument about why should we worried about AGI.”

I see many, perhaps the most simple and most convincing is the mere question of “what happens when AI becomes better than humans at coding?”, aka it can re-write its own code, aka it becomes totally unpredictable.

“Quite the contrary, building AGI is still an active area of research with no clear solution.”

There was also no clear solution to many other transformative technologies a couple years before their invention, from atomic bombs to airplanes. With both these examples, many, if not most specialists also said they were impossible.

Either the world wakes up, or we’re screwed. No one can now for sure whether AGI will come soon or even if it’s possible. But the high probability of it should make anyone scared. And even higher is the probability that if AI progress continues we’ll end up with dangerous AI, not necessarily AGI. Aka something difficult or impossible to control.

superads91 15 Mar 2022 10:16 UTC
6 points
in reply to: Dojan’s comment on: What’s, in your opinion, the real reason for this war?
“This is only my take, apply salt as needed. I am but a lone keyboard warrior in a big big sea of history, propaganda and geopolitics.”

It was a good one. But the main problems come, indeed, from its central metaphor, which flaws (which you acknowledge) can’t be ignored.

The US can’t be compared to the Warsaw Pact and the USSR. US states don’t have any history pre-US (not civilization-wise at least, and their former claimers have been ethnically anihilated too), whereas all of these Warsaw pact nations had a clear history outside of the connection to the Warsaw Pact and the USSR. A clear distinct identity, on all levels.

What is also obvious but can’t be ignored is the mere differences between NATO and Warsaw Pact. I’ll start by saying that NATO is far from perfect and the West is far from being free from corruption, but compared to the Russian dominance, they’re heaven. Democracy, human rights, economic development, freedom. It’s understandable that nations want to join NATO, but don’t want to join the Warsaw Pact or ally with modern Russia. They are seduced into the former, but coerced into the latter. That’s why you can’t compare NATO expanding West, with Warsaw Pact expanding East.

So I don’t really think that that comparison leads anywhere, nor do I think that Putin and the regime are naive enough to actually believe in it, aka to believe in their own propaganda. To pull a few heartstrings, that’s much more plausible. But then, my doubt still stands whether those heartstrings paint the full picture.

“Now, that being said, I don’t agree with the view of NATO as purely a neutral defense pact. It in fact places clearly ideological conditions on it’s members, some of which are overtly political: (emphasis mine) “Countries aspiring for NATO membership are also expected to meet certain political, economic and military goals in order to ensure that they will become contributors to Alliance security as well as beneficiaries of it.”″

Well, neutral it isn’t. It certainly is pro-Western values, and its members must have such basic values like democracy, as well as some basic economic and military power “to ensure that they will become contributors to Alliance security as well as beneficiaries of it” (self-explanatory). Doesn’t mean it wants to invade the world’s greatest nuclear power (aka commit planetary suicide).

But then, there’s once again the question whether these basic Western values of democracy and human rights are really specifically Western, or whether only the West has been fortunate enough to develop the political structure/ability to implement them. Are most Russians and Chinese, deep down, pro “Western” values? I believe they are. No wonder “everyone” wants to join NATO! Poor people just want freedom and protection. If that’s expansionism, then I’m all for it. (Still doesn’t equate to the suicidal act of invading the world’s greatest nuclear power because we don’t like them.)

superads91 5 Feb 2022 14:36 UTC
6 points
on: February Open Thread
Hey everyone. I’m new here. I’ve recently been kinda freaking out about AGI and its timelines… Specially after reading Eliezer’s post about there being no fire-alarm.

However, he also mentions that “one could never be sure of a new thing coming, except for the last one or two years” (something along those lines).

So, in your opinion, are we already at a stage where AGI could arrive anytime? Because due to things like GPT-3, Wu Dao 2.0 and AlphaCode, I’ve been getting really scared… Plus if there is something more advanced being developed in secret...

Or will there at least be a 1-2 year “last epistemic stage” which we can be sure we haven’t reached yet? (as Soares also mentions)

Cause everyday I’ve been looking out the window expecting the nano-swarms to come lol… But I’m just a lay person, so I’d like to hear some more expert opinions.

superads91 3 Apr 2022 23:22 UTC
5 points
in reply to: orthonormal’s comment on: MIRI announces new “Death With Dignity” strategy
I meant to reply to the OP, not Eli.

superads91 3 Apr 2022 7:13 UTC
4 points
in reply to: birdy’s comment on: MIRI announces new “Death With Dignity” strategy
“what I really don’t understand is why ‘failure to solve the problem in time’ sounds so much like ‘we’re all going to die, and that’s so certain that some otherwise sensible people are tempted to just give in to despair and stop trying at all’ ”

I agree. In this community, most people only talk of x-risk (existential risk). Most people equate failure to align AI to our values to human extinction. I disagree. Classic literature examples of failure can be found, like With Folded Hands, where AI creates an unbreakable dictatorship, not extinction.

I think it’s for the sake of sanity (things worse than extinction are quite harder to accept), or not to scare the normies, who are already quite scared.

But it’s also true that unaligned AI could result in a kinda positive outcome, or even neutral. I just personally wouldn’t put much probability in there. Why? 2 concepts that you can look up on Lesswrong: orthogonality thesis (high intelligence isn’t necessarily correlated to high values), and basic AI drives (advanced AI would naturally develop dangerous instrumental goals like survival and resource acquisition). And also that it’s pretty hard to tell computers to do what we mean, which scaled up could turn out very dangerous.

(See Eliezer’s post “Failed Utopia 4-2”, where an unaligned AGI ends up creating a failed utopia which really doesn’t sound THAT bad, and I’d say is even much better than the current world when you weight all the good and bad.)

Fundamentally, we just shouldn’t take the gamble. The stakes are too high.

If you wanna have an impact, AI is the way to go. Definitely.

superads91 2 Apr 2022 9:31 UTC
4 points
on: MIRI announces new “Death With Dignity” strategy
Personally I don’t blame it that much on people (that is, those who care), because maybe the problem is simply intractable. This paper by Roman Yalmpolskiy is what has convinced me the most about it:

https://philpapers.org/rec/YAMOCO

It basically asks the question: is it really possible to be able to control something much more intelligent than ourselves and which can re-write its own code?

Actually I wanna believe that it is, but we’d need something on the miracle level, as well as way more people working on it. As well as way more time. It’s virtually impossible in a couple decades and with a couple hundred researchers, i.e. intractable as things stand.

That leaves us with political solutions as the only tangible solutions on the table. Or also technical contingency measures, which could perhaps be much easier to develop than alignment, and prevent the worst outcomes.

Speaking of which, I know we all wanna stay sane, but death isn’t even the worse possible outcome. And this is what makes the problem so pressing, so much more than nuclear risk or grey goo. If we could indeed die with dignity at least, it wouldn’t be so bad already. (I know this sounds extremely morbid, but like Eliezer, I’m beyond caring.)

superads91 15 Mar 2022 15:08 UTC
4 points
in reply to: Dojan’s comment on: What’s, in your opinion, the real reason for this war?
Of course I’m aware of the geopolitical board game. I never doubted that Russia had a lot to gain with gaining back the pre-2014 Ukrainian loyalty. Or that Ukraine’s democracy threatens Putin’s regime. Or that such things are immoral (from Russia). Or that this war is immoral.

My only doubts come from the fact that I really don’t see this war a net-gain for Russia in those regards. It has nothing to do with moral shock or being oblivious about the geopolitical game. It is, indeed, a geopolitical consideration.

“Either way, invasion is not the way in which NATO threatens Putin.”

True, the threat of having a protected democracy on his doorstep is a potential political threat to his regime. But not eminent, and he won’t live forever. That’s why, while in other comments I’ve considered it one of the strongest “real motives” to me, it still feels insufficient.

We can be against Western expansionism as much as we want, but without it, half the world would be living much worse lives, from North America, to Australia, to Japan and South Korea (cultural expansionism), to even the rest of Asia who have also benefited from such cultural expansionism. Also, if the West hadn’t expanded this much, both culturally and politically, our way of life today would be in much more danger. We would stand a lot less strong today. Perhaps wouldn’t even exist as liberal democracies.

One thing is to try to do it while avoiding (nuclear) war. But to be against it is naive. Everybody wants to rule the world. It’s better if the good guys do it.

superads91 14 Mar 2022 22:59 UTC
4 points
in reply to: Nanda Ale’s comment on: What’s, in your opinion, the real reason for this war?
To be honest, all 3 look extremely likely, in fact much more likely than all other hypothesis I’ve ever heard, both on this thread and elsewhere. It’s always about money and power, way more than nationalism or a bunch of yes-men around Putin. The things that some humans are willing to do for power… Thanks

superads91 5 Feb 2022 16:08 UTC
4 points
in reply to: Anirandis’s comment on: Paperclippers, s-risks, hope
Thanks for the attentious commentary.
1. Yeah, I was guessing that the smiley faces wouldn’t be the best example… I was just wanting to draw something from the Eliezer/Bostrom universe since I had mentioned the paperclipper beforehand. So, maybe a better Eliezer-Bostrom example would be, we ask the AGI to “make us happy”, and it puts everyone paralyzed in hospital beds on dopamine drips. It’s not hard to think that after a couple hours of a good high, this would actually be a hellish existence, since human happiness is way more complex than the amount of dopamine in one’s brain (but of course, Genie in the Lamp, Mida’s Touch, etc)
2. So, don’t you equate this kind of scenario with a significant amount of suffering? Again, forget the bad example of the smiley faces, and reconsider. (I’ve actually read in a popular lesswrong post about s-risks Paul clearly saying that the risk of s-risk was 1/100th of the risk of x-risk (which makes for even less than 1/100th overall). Isn’t that extremely naive, considering the whole Genie in the Lamp paradigm? How can we be so sure that the Genie will only create hell 1 time for each 100 times it creates extinction?)
3. a) I agree that a suffering-maximizer is quite unlikely. But you don’t necessarily need one to create s-risks scenarios. You just need a Genie in the Lamp scenario. Like the dopamine drip example, in which the AGI isn’t trying to maximize suffering, quite on contrary, but since it’s super-smart in Sciences but lacks human common sense (a Genie), it ends up doing it.
b) Yes I had read that article before. While it presents some fair solutions, I think it’s far from being mostly solved. “Since hyperexistential catastrophes are narrow special cases (or at least it seems this way and we sure hope so), we can avoid them much more widely than ordinary existential risks.” Note the “at least it seems this way and we surely hope so”. Plus, what’s the odds that the first AGI will be created by someone who listens to what Eliezer has to say? Not that bad actually, if you consider US companies, but if you consider China, then dear God...

On your PS1, yeah definitely not willing to do cryonics, and again, s-risks don’t need to come from threats, just misalignment.

Sorry if I black pilled you with this, maybe there is no point… Maybe I’m wrong. I hope I am.

superads91 8 Apr 2022 11:10 UTC
3 points
in reply to: ChristianKl’s comment on: MIRI announces new “Death With Dignity” strategy
Everyone knows that the Holocaust wasn’t just genocide. It was also torture, evil medical experiments, etc. But you’re right, I should have used a better example. Not that I think that anyone really misunderstood what I meant.

superads91 16 Mar 2022 11:19 UTC
3 points
in reply to: Dojan’s comment on: What’s, in your opinion, the real reason for this war?
“Just that I haven’t yet found any compelling evidence far outside of it. If you feel like you have some, I’d be most interested.”

8.) Nanda Ale’s comment. Which, in my opinion, isn’t the whole portion of the Real Reason (tm) either, but paints the largest portion of it so far, in my very personal and subjective opinion.

superads91 14 Mar 2022 22:43 UTC
3 points
in reply to: Arcayer’s comment on: What’s, in your opinion, the real reason for this war?
Very interestingly put, haha!

“Ukraine shot a plane down, and the west blames Russia. Do the Russians swallow their pride and put up with this, or do they escalate? And of course they escalate.”

And to think the assassination of some Archduke was a petty starting event for a world war...

(No sarcasm, the world is indeed stupid, like that.)

superads91 1 Mar 2022 16:05 UTC
3 points
in reply to: ztzuliios’s comment on: Russia has Invaded Ukraine
Maybe no need to attribute any qualities to those papers. If, for instance, the situation was such that such war was inevitable, then yes, it makes sense to know if we could “survive” someway.

My claim was simply that NATO would never invade Russia knowing that it would take at least civilization collapsed. It’s completely self-defeating. The person to which I responded said “it’s not as simple as”Russia has nukes, the end”, in the context of a possible NATO invasion of Russia. All I meant to say was it effectively is.

superads91 28 Feb 2022 14:22 UTC
3 points
in reply to: Viliam’s comment on: Russia has Invaded Ukraine
NATO is a defensive organization. Article 5 says “an attack on a member is an attack on all members”. So far is was only activated once, due to the 9/11 attacks. Yet the US has been in many wars since it joined NATO.

superads91 10 Feb 2022 0:51 UTC
3 points
in reply to: Mitchell_Porter’s comment on: Can the laws of physics/nature prevent hell?
Basically we need a lot more Eliezers. We need a lot more AI advocates that tell it like it is, that make us shit our pants, that won’t soften their message to appear reasonable. That are actually realistic about timelines and risk. As long as most of the popular advocates keep with the approach of “don’t panic, don’t be afraid, don’t worry, it’s doable, if only we remain positive and believe in ourselves” then there is no hope. As long as people will keep lying to themselves to avoid panic, there is no hope. Panic can be treated in many ways, even in the most extreme cases with benzodiazepines. Disaster once it settles has no solution, and it’s a lot worse.

superads91

[un­ti­tled post]

[untitled post]