skeptical_lurker

Karma: 2,790

skeptical_lurker Nov 12, 2016, 5:31 PM
1 point
in reply to: entirelyuseless’s comment on: Yudkowsky vs Trump: the nuclear showdown.
Well, its possible that e.g.

Conservative judges → ban abortion → increased crime → government spends more energy trying to stop crime and less on FAI reserch → paperclips

But we’re into the realm of tiny minute one-in-a-million probabilities here. Altering Supreme Court appointments is not exactly the most effective way to fight x-risk by any stretch of the imagination.

skeptical_lurker Nov 12, 2016, 5:24 PM
3 points
in reply to: siIver’s comment on: Yudkowsky vs Trump: the nuclear showdown.

I think c/c is likely to kill several billion people overall before we stop it, and might also lead to extinction.

Scientific consensus is that warming under 2 degrees c could be good, warming over that is bad. Worst case scenario is that we will hit that around 2060. Many forecast the singularity as most likely to happen in the 2040s, but even if that is over optimistic, solar panels are halving in cost per watt each decade. Naively extrapolating, by 2060 solar power should be 20x cheaper, so even if the singularity is delayed we should still be able to move most of the economy onto clean energy even without subsidies long before we reach dangerous levels of warming.

I certainly think that global warming is a risk we should monitor, and keep funding research into solar panels, but I really don’t think it needs to be a priority.

skeptical_lurker Nov 12, 2016, 5:15 PM
6 points
in reply to: ChristianKl’s comment on: Yudkowsky vs Trump: the nuclear showdown.
I’m not denying that Trump does not seem like the best choice (out of the entire US population) for geopolitics. I’ll concede that point. But look at Hillary—doesn’t she want to impose a no-fly zone over Syria? Threaten to shoot down Russian planes? Why? This isn’t the cold war, where maybe we had to draw a line in the sand. If the Russians want to have greater influence over Syria, let them.

Maybe there is another level to this that I don’t understand because I am not an expert in geopolitics, but I would have thought it wiser to not restart the cold war, let Russia have influence over some nearby countries, and present a united front against radical Islam.

As for the economics arguments, to be frank I attach little weight to what the media says given how biased they’ve been. Lets look at the markets, where people actually put their money on the line. The USD is up against the Euro and the Yen. The sp500 is up 1%. I see no reason for the US to worry about economics.

Finally, Trump will have advisers. And even if he does appear a little unstable, well, there are some games (like chicken) where someone that appears to be a little crazy will beat a calm rational person running causal decision theory every time.

skeptical_lurker Nov 12, 2016, 8:35 AM
1 point
in reply to: siIver’s comment on: Yudkowsky vs Trump: the nuclear showdown.
Elaborate? Even with just a link?

skeptical_lurker Nov 12, 2016, 8:35 AM
6 points
in reply to: siIver’s comment on: Yudkowsky vs Trump: the nuclear showdown.
Re Climate change, I agree with Kawoomba, with the caviat that GW could provoke conflict which causes an indirect X-risk.

Interestingly, the Green party candidate said Clinton is worse for nuclear war. Maybe she’s wrong, but I don’t think the issue is as obvious as you think it is.

Moreover, the Supreme Court nominees probably have indirect consequences on c/c aswell.

I though the Supreme Court dealt more with civil rights stuff. How will they affect CC?

skeptical_lurker Nov 11, 2016, 8:10 PM
5 points
in reply to: ChristianKl’s comment on: Yudkowsky vs Trump: the nuclear showdown.

That’s not the only argument. The fact that Trump suggested that it’s fine that the Saudi’s (and others) get nukes. That statement gave him no electoral advantage but he still made it.

That is pretty disturbing. I wish people would lead with ‘Trump is ok with nuclear proliferation’ rather than ‘Trump is basically Hitler’.

I don’t think that’s the case. I doubt EY would have made the same arguments against Ron Paul.

Back to the problem that ‘right/left’ is too simplistic. Perhaps I should have said libertarians/progressives vs authoritarians/conservatives, but since neither candidate is libertarian I was not thinking in those terms.

Trump got rich because he frequently didn’t pay people what he owned them. The the campaign he suggested that he wants to do the same with US debt.

I recall that there is an (as yet unresolved?) court case about him failing to pay people, but to say that this is the main cause of his wealth sounds like a stretch. Increasing the national debt is also worrying, but (a) Hillary’s economic policies she would presumably raise the debt too albeit to a lesser extent, and (b) we’ve now moved off talking about nuclear war, at least directly
What links here?
- ChristianKl's comment on Open thread, Nov. 14 - Nov. 20, 2016 by MrMind (Nov 19, 2016, 1:22 AM; 5 points)

skeptical_lurker Nov 11, 2016, 7:33 PM
7 points
in reply to: siIver’s comment on: Yudkowsky vs Trump: the nuclear showdown.

Well, I agree with it. Yudkowksy and S/A both seem to view the world through the lense of “let’s do everything possible to a) reach singularity and b) get singularity right” which I think is the only rational perspective based on their beliefs about singularity (and utilitarianism). The amount of value associated with the singularity makes everything else insignificant in comparison.

Well, I agree with that. The question is, does Trump as president increase the probability of human extinction, and why? Bear in mind that Peter Theil, who has donated a lot of money to MIRI, supports Trump, so its not as clear-cut as all the smart people being on the same side of this issue.

skeptical_lurker Nov 11, 2016, 6:23 PM
7 points
in reply to: siIver’s comment on: Yudkowsky vs Trump: the nuclear showdown.

I would like this to be a comment on methodology, about if their arguments were sound given what they knew and believed. I most definitely do not want this to decay in a lamentation about the results

So… any comments on methodology?

skeptical_lurker Nov 11, 2016, 5:02 PM
16 points
on: Yudkowsky vs Trump: the nuclear showdown.
EY’s central argument for level B incompetence was that Trump is creating ambiguity around which countries the US will defend against Russia, which could lead to war. Now, I agree that it would be wrong for a sitting president to create that ambiguity, but a presidential candidate has to ask those questions, otherwise the foreign policy can never change. As long as Trump arrives at a concrete policy over which countries the US will defend when he becomes president, I don’t see that there is a problem.

I also don’t see that the status quo is keeping the world all that safe, with a proxy war between the US and Russia in Syria.

Most substantial variations from the equilibrium are disasters, and if you put a high-variance candidate, someone whose main point is to subvert the status quo, in charge, then with overwhelming probability you’re headed off to a cliff.

This is exactly the same as the argument for radical conservatism that the neoreactionaries make. Can you really believe that when the right challenges the status quo, priors are that they almost certainly wrong, but when progressives challenge the status quo they are almost certainly correct? This is extremely motivated reasoning and totally inconsistent.

And EY is all about finding the ‘correct contrarians’ and subverting the status quo on every other issue.

skeptical_lurker Oct 15, 2016, 1:35 PM
0 points
in reply to: Lumifer’s comment on: Open thread, Oct. 10 - Oct. 16, 2016
Indeed, but I was wondering whether modern social and technological changes will accelerate this.

skeptical_lurker Oct 15, 2016, 1:20 PM
1 point
on: Barack Obama’s opinions on near-future AI [Fixed]

Then there could be an algorithm that said, “Go penetrate the nuclear codes and figure out how to launch some missiles.” If that’s its only job, if it’s self-teaching and it’s just a really effective algorithm, then you’ve got problems. I think my directive to my national security team is, don’t worry as much yet about machines taking over the world. Worry about the capacity of either nonstate actors or hostile actors to penetrate systems, and in that sense it is not conceptually different than a lot of the cybersecurity work we’re doing.

Please tell me this isn’t an actual possibility. Surely nuclear launch must rely on multi-factor authentication with one-time-pads and code phrases in sealed, physical envelopes. A brain the size of a planet could not break a one-time pad. I know a superhuman AI could probably hack the net, but please tell me that nuclear missiles are not connected to the internet.

But… Obama must know the capacity of America’s nuclear security. The best reason I can think for him to raise this possibility is to confuse America’s enemies into thinking that the nuclear weapons are not properly secured, so that they will attack the nuclear launch codes which are actually secure, rather than attempting a more low-tech attack like another September 11.

skeptical_lurker Oct 15, 2016, 1:08 PM
0 points
in reply to: DanArmak’s comment on: Barack Obama’s opinions on near-future AI [Fixed]
Time to Godwin myself:

1930′s Germany: The problem with relativity is that it’s developed by Jews. We need an ethnically pure physics.

2010′s USA : The problem with AI is that it’s developed by white men. We need an ethnically diverse compsci.

skeptical_lurker Oct 10, 2016, 6:26 PM
10 points
in reply to: turchin’s comment on: Open thread, Oct. 10 - Oct. 16, 2016
Ignore all the stuff about provably friendly AI, because AFAIK its fairly stuck at the fundamental level of theoretical impossibility due to lob’s theorem and its prob going to take a lot more than five years. Instead, work on cruder methods which have less chance of working but far more chance of actually being developed in time. Specifically, if Google are developing it in 5 years, then its probably going to be deepmind with DNNs and RL, so work on methods that can fit in with that approach.

skeptical_lurker Oct 10, 2016, 6:21 PM
6 points
in reply to: Lumifer’s comment on: Open thread, Oct. 10 - Oct. 16, 2016
That doesn’t mean that there is nothing to do—if you don’t know what FAI is, then you try to work out what it is.

skeptical_lurker Oct 10, 2016, 6:14 PM
3 points
on: Open thread, Oct. 10 - Oct. 16, 2016
We live in an increasingly globalised world, where moving between countries is both easier in terms of transport costs and more socially acceptable. Once translation reaches near-human levels, language barriers will be far less of a problem. I’m wondering to what extent evaporative cooling might happen to countries, both in terms of values and economically.

I read that France and Greece lost 3 & 5% of their millionaires last year (or possibly the year before), citing economic depression and rising racial/religious tension, with the most popular destination being Australia (as it has the 1st or 2nd highest HDI in the world). 3-5% may not seem like a lot, but if it were sustained for several years it quickly piles up. The feedback effects are obvious—the wealthier members of society find it easier to leave and perhaps have more of a motive to leave an economic collapse, which decreases tax revenue, which increases collapse etc. On the flip side, Australia attracts these people and its economy grows more making it even more attractive...

Socially, the same effect as described in EY’s essay I linked happens on a national scale—if the ‘blue’ people leave, the country becomes ‘greener’ which attracts more greens and forces out more blues. And social/economic factors feed into each other too—economic collapses cause extremism of all sorts, while I imagine a wealthy society attracting elites would be more able to handle or avoid conflicts.

Now, this is not automatically a bad thing, or at least it might be bad locally for some people, but perhaps not globally. Any thoughts as to what sort of outcomes there might be? And incidentally, how many people can you fit in Australia? I know its very big, but also has a lot of desert.

skeptical_lurker Oct 10, 2016, 5:55 PM
−2 points
in reply to: ZankerH’s comment on: Open thread, Oct. 10 - Oct. 16, 2016
Google do not strike me as incompitant, and they do have ethics oversite for AI. Worry, yes, despair, no.

skeptical_lurker 6 Oct 2016 21:15 UTC
2 points
in reply to: ChristianKl’s comment on: Open thread, Oct. 03 - Oct. 09, 2016
It depends whether we are using ‘racist’ to mean ‘believes that some races are superior to others in certain respects’ or ‘has less empathy for other races’. In the first case, sure, maybe you would date someone of another race, because group differences aren’t so important when dealing with individuals. But in the latter case… if you are less able to empathise with people of other races it would seem really weird to date them.

skeptical_lurker 6 Oct 2016 21:06 UTC
0 points
in reply to: ChristianKl’s comment on: Open thread, Oct. 03 - Oct. 09, 2016

If a AGI does thinks in a very complicated way it might not meaningfully get consent for anything because it can’t explain it’s reasoning to humans.

Is that necessary for consent? I mean, one does not have to understand the rationale for undergoing a medical procedure in order to consent to it. Its more important to know the potential risks.

skeptical_lurker 5 Oct 2016 19:11 UTC
2 points
in reply to: Brillyant’s comment on: Open thread, Oct. 03 - Oct. 09, 2016

I don’t think it means that. I don’t think she meant that.

I’m pretty sure that is what she means. There is a big controversy in the US over whether the police are racist, not over whether the police have cognitive biases. I would be overjoyed if presidential candidates really were discussing cognitive biases.

My understanding is that humans have a tribal in/out group mentality that may use race as way to classify other humans as “others”. They can also use religion, class, culture, etc.

No disagreement here.

skeptical_lurker 5 Oct 2016 18:53 UTC
1 point
in reply to: ChristianKl’s comment on: Open thread, Oct. 03 - Oct. 09, 2016

How is a AGI supposed to optimize for values that aren’t consistent?

I am not saying this is a trivial problem, but it is a separate problem from ‘the hidden complexity of wishes’ problem.

Does that mean that the AGI should start doing genetic manipulation that prevents people from being gay?

Well, if the CEV of the anti-gay, pro-genetic manipulation people exceeds the CEV of the pro-gay/anti-genetic manipulation people then I suppose it would, although I’m not sure whether your question means genetic manipulation with or without consent (also, if a gay person wants to be straight, some would say that should be banned, so consent cuts both ways), and so you also have to take into account the CEV on the issue of consent. Its also true that a super intelligence might be able to talk someone into consenting to almost anything.

Yes, a CEV FAI would forcibly alter people’s sexualities if the aggrigated preferences in favour of that were strong enough. A democratic system will be a tyranny of the majority if the majority are tyrants.

Is that what the person who made the claim means?

I dunno, since I’ve only heard one sentence from this hypothetical person. But I would imagine that this sort of person would probably think that genetic manipulation is playing god, and moreover that superintelligent AI is playing god. Their strongest wish might be for the AI to turn itself off.

EDIT: how to react to the god hates fags people also depends upon whether being anti gay is a terminal value to these people, or whether it is predicated upon the existance of god. I’m assuming the FAI would not beleive in god, but then again some people might have faith as a terminal value, so… its complicated.