Thanks for pointing to the orthogonality thesis as a reason for believing the chance would be low that advanced aliens would be nice to humans. I followed up by reading Bostrom’s “The Superintelligent Will,” and I narrowed down my disagreement to how this point is interpreted:
In a similar vein, even if there are objective moral facts that any fully rational agent would comprehend, and even if these moral facts are somehow intrinsically motivating (such that anybody who fully comprehends them is necessarily motivated to act in accordance with them) this need not undermine the orthogonality thesis. The thesis could still be true if an agent could have impeccable instrumental rationality even whilst lacking some other faculty constitutive of rationality proper, or some faculty required for the full comprehension of the objective moral facts. (An agent could also be extremely intelligent, even superintelligent, without having full instrumental rationality in every domain.)
Just because it’s possible that an agent could have impeccable instrumental rationality while lacking in epistemic rationality to some degree, I expect the typical case that leads to very advanced intelligence would eventually involve synergy between growing both in concert, as many here at Less Wrong are working to do. In other words, a highly competent general intelligence is likely to be curious about objective facts across a very diverse range of topics.
So while aliens could be instrumentally advanced enough to make it to Earth without having ever made basic discoveries in a particular area, there’s no reason for us to expect that it is specifically the area of morality where they will be ignorant or delusional. A safer bet is that they have learned at least as many objective facts as humans have about any given topic on expectation, and that a topic where the aliens have blind spots in relation to some humans is an area where they would be curious to learn from us.
A policy of unconditional harmlessness and friendliness toward all beings is a Schelling Point that could be discovered in many ways. I grant that humans may have it relatively easy to mature on the moral axis because we are conscious, which may or may not be the typical case for general intelligence. That means we can directly experience within our own awareness facts about how happiness is preferred to suffering, how anger and violence lead to suffering, how compassion and equanimity lead to happiness, and so on. We can also see these processes operating in others. But even a superintelligence with no degree of happiness is likely to learn whatever it can from humans, and learning something like love would be a priceless treasure to discover on Earth.
If aliens show up here, I give them at least a 50% chance of being as knowledgeable as the wisest humans in matters of morality. That’s ten times more than Yudkowsky gives them and perhaps infinitely more than Hotz does!
Hello friends. It’s hard for me to follow the analogies from aliens to AI. Why should we should expect harm from any aliens who may appear?
15:08 Hotz: “If aliens were to show up here, we’re dead, right?” Yudkowsky: “It depends on the aliens. If I know nothing else about the aliens, I might give them something like a five percent chance of being nice.” Hotz: “But they have the ability to kill us, right? I mean, they got here, right?” Yudkowsky: “Oh they absolutely have the ability. Anything that can cross interstellar distances can run you over without noticing—well, they would notice, but they wouldn’t ca—” [crosstalk] Hotz: “I didn’t expect this to be a controversial point. But I agree with you that if you’re talking about intelligences that are on the scale of billions of times smarter than humanity… yeah, we’re in trouble.”
Having listened to the whole interview, my best guess is that Hotz believes that advanced civilizations are almost certain to be Prisoner’s Dilemma defectors in the extreme, i.e. they have survived by destroying all other beings they encounter. If so, this is quite disturbing in connection with 12:08, in which Hotz expresses his hope that our civilization will expand across the galaxy (in which case we potentially get to be the aliens).
Hotz seems certain aliens would destroy us, and Eliezer gives them only a five percent chance of being nice.
This is especially odd considering the rapidly growing evidence that humans actually have been frequently seeing and sometimes interacting with a much more advanced intelligence.
It’s been somewhat jarring for my belief in the reality of nonhuman spacecraft to grow by so much in so little time, but overall it has been a great relief to consider the likelihood that another intelligence in this universe has already succeeded in surviving far beyond humankind’s current level of technology. It means that we too could survive the challenges ahead. The high-tech guys might even help us, whoever they are.
But Hotz and Yudkowsky seem to agree that seeing advanced aliens would actually be terrible news. Why?