It doesn’t really matter whether the AI uses their full computational capacity. If the AI has a 100000 times larger capacity (which is again a conservative lower bound) and it only uses 1% of it, it will still be 1000 as smart as the human’s full capacity.
AGI’s algorithm will be better, because it has instant access to more facts than any human has time to memorize, and it will not have all of the biases that humans have. The entire point of the sequences is to list dozens of ways that the human brain reliably fails.
“[the brain] is sending signals at a millionth the speed of light, firing at 100 Hz, and even in heat dissipation [...] 50000 times the thermodynamic minimum energy expenditure per binary swtich operation”
AI will be quantitatively smarter because it’ll be able to think over 10000 times faster (arbitrary conservative lower bound) and it will be qualitatively smarter because its software will be built by an algoirthm far better than evolution
“Less than a third of students by their own self-appointed worst-case estimate *1.”
missing a word here, I think.
I think your post is spot on.
re-live. Although I’d rather live the same amount of time from now onward.
First question: I know you admire Trump’s persuasion skills, but what I want to know is why you think he’s a good person/president etc.
Answer: [talks about Trump’s persuasion skills]
This is an exceptionally well reasoned article, I’d say. Particular props to the appropriate amount of uncertainty.
Well, if you put it like that I fully agree. Generally, I believe that “if it doesn’t work, try something else” isn’t followed as often as it should. There’s probably a fair number of people who’d benefit from following this article’s advice.
I don’t quite know how to make this response more sophisticated than “I don’t think this is true”. It seems to me that whether classes ore lone-wolf improvement is better is a pretty complex question and the answer is fairly balanced, though overall I’d give the edge to lone-wolf.
I don’t know what our terminal goals are (more precisely than “positive emotions”). I think it doesn’t matter insofar as the answer to “what should we do” is “work on AI alignment” either way. Modulo that, yeah there are some open questions.
On the thesis of suffering requiring higher order cognition in particular, I have to say that sounds incredibly implausible (for I think fairly obvious reasons involving evolution).
This looks solid.
Can you go into a bit of detail on the level / spectrum of difficulty of the courses you’re aiming for, and the background knowledge that’ll be expected? I suspect you don’t want to discourage people, but realistically speaking, it can hardly be low enough to allow everyone who’s interested to participate meaningfully.
Yeah, you’re of course right. In the back of my mind I realized that the point I was making was flawed even as I was writing it. A much weaker version of the same would have been correct, “you should at least question whether your intuition is wrong.” In this case it’s just very obvious to me me that there is nothing to be fixed about utilitarianism.
Anyway, yeah, it wasn’t a good reply.
This is the ultimate example of… there should be a name for this.
You figure out that something is true, like utilitarianism. Then you find a result that seems counter intuitive. Rather than going “huh, I guess my intuition was wrong, interesting” you go “LET ME FIX THAT” and change the system so that it does what you want...
man, if you trust your intuition more than the system, then there is no reason to have a system in the first place. Just do what is intuitive.
The whole point of having a system like utilitarinism is that we can figure out the correct answers in an abstarct, general way, but not necessarily for each particular situation. Having a system tells us what is correct in each situation, not vice versa.
The utility monster is nothing to be fixed. It’s a natural consequence of doing the right thing, that just happens to make some people uncomfortable. It’s hardly the only uncomfortable consequence of utilitarianism, either.
This seems like something we should talk about more.
Although, afaik there shouldn’t be a decision between motivation selection and capability controlling measures – the former is obviously the more important part, but you can also always “box” the AI in addition (insofar as that’s compatible with what you want it to do).
That sounds dangerously like justifying inaction.
Literally speaking, I don’t disagree. It’s possible that spreading awareness has a net negative outcome. It’s just not likely. I don’t discourage looking into the question, and if facts start pointing the other way I can be convinced. But while we’re still vaguely uncertain, we should act on what seems more likely right now.
I guess it’s a legit argument, but it doesn’t have the research aspect and it’s a sample size of one.
This just seems like an incredibly weak argument to me. A) it seems to me that prior research will be influenced much more than the probability for an arms race, because the first is more directly linked to public perception, B) we’re mostly trying to spread awareness of the risk not the capability, and C) how do we even know that more awareness on the top political levels would lead to a higher probability for an arms race, rather than a higher probability for an international cooperation?
I feel like raising awareness has a very clear and fairly safe upside, while the downside is highly uncertain.
Pretty sure it is. You have two factors, increasing the awareness of AI risk and of AI specifically. The first is good, the second may be bad but since the set of people caring about AI generally is so much larger, the second is also much less important.
I whole-heartedly agree with you, but I don’t have anything better than “tell everyone you know about it.” On that topic, what do you think is the best link to send to people? I use this, but it’s not ideal.
Q: Evolution is a dumb algorithm, yet it produced halfway functional minds. How can it be that the problem isn’t easy for humans, who are much smarter than evolution?
A: Evolution’s output is not just one functional mind. Evolution put out billions of different minds, an extreme minority of them being functional. If we had a billion years of time and had a trillion chances to get it right, the problem would be easy. Since we only have around 30 years and exactly 1 chance, the problem is hard.