siIver

Karma: 161

siIver 1 Nov 2017 10:30 UTC
0 points
in reply to: curi’s comment on: Questions about AGI’s Importance
It doesn’t really matter whether the AI uses their full computational capacity. If the AI has a 100000 times larger capacity (which is again a conservative lower bound) and it only uses 1% of it, it will still be 1000 as smart as the human’s full capacity.

AGI’s algorithm will be better, because it has instant access to more facts than any human has time to memorize, and it will not have all of the biases that humans have. The entire point of the sequences is to list dozens of ways that the human brain reliably fails.

siIver 1 Nov 2017 9:32 UTC
0 points
on: Questions about AGI’s Importance
Because

“[the brain] is sending signals at a millionth the speed of light, firing at 100 Hz, and even in heat dissipation [...] 50000 times the thermodynamic minimum energy expenditure per binary swtich operation”

https://www.youtube.com/watch?v=EUjc1WuyPT8&t=3320s

AI will be quantitatively smarter because it’ll be able to think over 10000 times faster (arbitrary conservative lower bound) and it will be qualitatively smarter because its software will be built by an algoirthm far better than evolution

siIver 9 Sep 2017 11:44 UTC
0 points
on: Instrumental Rationality Sequence Finished! (w/ caveats)
“Less than a third of students by their own self-appointed worst-case estimate *1.”

missing a word here, I think.

siIver 9 Sep 2017 10:46 UTC
0 points
on: Inconsistent Beliefs and Charitable Giving
I think your post is spot on.

siIver 31 Aug 2017 22:56 UTC
1 point
on: Is life worth living?
re-live. Although I’d rather live the same amount of time from now onward.

siIver 20 Jul 2017 19:41 UTC
8 points
on: Sam Harris and Scott Adams debate Trump: a model rationalist disagreement
First question: I know you admire Trump’s persuasion skills, but what I want to know is why you think he’s a good person/president etc.

Answer: [talks about Trump’s persuasion skills]

Yeah, okay.

siIver 9 Jul 2017 14:18 UTC
0 points
on: Daniel Dewey on MIRI’s Highly Reliable Agent Design Work
This is an exceptionally well reasoned article, I’d say. Particular props to the appropriate amount of uncertainty.

siIver 7 Jul 2017 20:52 UTC
2 points
in reply to: cousin_it’s comment on: Against lone wolf self-improvement
Well, if you put it like that I fully agree. Generally, I believe that “if it doesn’t work, try something else” isn’t followed as often as it should. There’s probably a fair number of people who’d benefit from following this article’s advice.

siIver 7 Jul 2017 20:41 UTC
0 points
on: Against lone wolf self-improvement
I don’t quite know how to make this response more sophisticated than “I don’t think this is true”. It seems to me that whether classes ore lone-wolf improvement is better is a pretty complex question and the answer is fairly balanced, though overall I’d give the edge to lone-wolf.

siIver 4 Jul 2017 20:23 UTC
1 point
on: We need a better theory of happiness and suffering
I don’t know what our terminal goals are (more precisely than “positive emotions”). I think it doesn’t matter insofar as the answer to “what should we do” is “work on AI alignment” either way. Modulo that, yeah there are some open questions.

On the thesis of suffering requiring higher order cognition in particular, I have to say that sounds incredibly implausible (for I think fairly obvious reasons involving evolution).

siIver 15 Jun 2017 19:30 UTC
1 point
on: Announcing AASAA—Accelerating AI Safety Adoption in Academia (and elsewhere)
This looks solid.

Can you go into a bit of detail on the level / spectrum of difficulty of the courses you’re aiming for, and the background knowledge that’ll be expected? I suspect you don’t want to discourage people, but realistically speaking, it can hardly be low enough to allow everyone who’s interested to participate meaningfully.

siIver 10 Jun 2017 15:01 UTC
1 point
in reply to: AlexMennen’s comment on: Destroying the Utility Monster—An Alternative Formation of Utility
Yeah, you’re of course right. In the back of my mind I realized that the point I was making was flawed even as I was writing it. A much weaker version of the same would have been correct, “you should at least question whether your intuition is wrong.” In this case it’s just very obvious to me me that there is nothing to be fixed about utilitarianism.

Anyway, yeah, it wasn’t a good reply.

siIver 8 Jun 2017 16:01 UTC
4 points
−2
on: Destroying the Utility Monster—An Alternative Formation of Utility
This is the ultimate example of… there should be a name for this.

You figure out that something is true, like utilitarianism. Then you find a result that seems counter intuitive. Rather than going “huh, I guess my intuition was wrong, interesting” you go “LET ME FIX THAT” and change the system so that it does what you want...

man, if you trust your intuition more than the system, then there is no reason to have a system in the first place. Just do what is intuitive.

The whole point of having a system like utilitarinism is that we can figure out the correct answers in an abstarct, general way, but not necessarily for each particular situation. Having a system tells us what is correct in each situation, not vice versa.

The utility monster is nothing to be fixed. It’s a natural consequence of doing the right thing, that just happens to make some people uncomfortable. It’s hardly the only uncomfortable consequence of utilitarianism, either.

siIver 26 May 2017 14:23 UTC
0 points
on: Existential risk from AI without an intelligence explosion
This seems like something we should talk about more.

Although, afaik there shouldn’t be a decision between motivation selection and capability controlling measures – the former is obviously the more important part, but you can also always “box” the AI in addition (insofar as that’s compatible with what you want it to do).

siIver 23 May 2017 16:32 UTC
0 points
in reply to: whpearson’s comment on: AGI and Mainstream Culture
That sounds dangerously like justifying inaction.

Literally speaking, I don’t disagree. It’s possible that spreading awareness has a net negative outcome. It’s just not likely. I don’t discourage looking into the question, and if facts start pointing the other way I can be convinced. But while we’re still vaguely uncertain, we should act on what seems more likely right now.

siIver 22 May 2017 18:47 UTC
0 points
in reply to: whpearson’s comment on: AGI and Mainstream Culture
I guess it’s a legit argument, but it doesn’t have the research aspect and it’s a sample size of one.

siIver 21 May 2017 18:32 UTC
1 point
on: AGI and Mainstream Culture
This just seems like an incredibly weak argument to me. A) it seems to me that prior research will be influenced much more than the probability for an arms race, because the first is more directly linked to public perception, B) we’re mostly trying to spread awareness of the risk not the capability, and C) how do we even know that more awareness on the top political levels would lead to a higher probability for an arms race, rather than a higher probability for an international cooperation?

I feel like raising awareness has a very clear and fairly safe upside, while the downside is highly uncertain.

siIver 17 May 2017 20:50 UTC
0 points
in reply to: whpearson’s comment on: Reaching out to people with the problems of friendly AI
Pretty sure it is. You have two factors, increasing the awareness of AI risk and of AI specifically. The first is good, the second may be bad but since the set of people caring about AI generally is so much larger, the second is also much less important.

siIver 16 May 2017 20:08 UTC
1 point
on: Reaching out to people with the problems of friendly AI
I whole-heartedly agree with you, but I don’t have anything better than “tell everyone you know about it.” On that topic, what do you think is the best link to send to people? I use this, but it’s not ideal.

siIver 13 May 2017 16:50 UTC
0 points
on: Why do we think most AIs unintentionally created by humans would create a worse world, when the human mind was designed by random mutations and natural selection, and created a better world?
Essentially:

Q: Evolution is a dumb algorithm, yet it produced halfway functional minds. How can it be that the problem isn’t easy for humans, who are much smarter than evolution?

A: Evolution’s output is not just one functional mind. Evolution put out billions of different minds, an extreme minority of them being functional. If we had a billion years of time and had a trillion chances to get it right, the problem would be easy. Since we only have around 30 years and exactly 1 chance, the problem is hard.