This fleshes out quite gingerly why orthogonality deniers are operating on faith rather reason. Orthogonality doesn’t say that a superintelligence necessarily has values incompatible with those of humans, only that it can. Orthogonality deniers must argue that under no circumstances can the values of an ASI be misaligned with those of humans. How can that be? They never provide a mechanism for this convergence toward human values. Moreover, you see counterexamples all the time: serial killers with high IQs that murder other people to satisfy base desires. When you hear these killers articulate their reasons for killing, they often evince values that are incompatible with valuing human life and dignity even as their minds are razor sharp. They know what is right, but they do otherwise because they want to; society has to carefully lock them up or put them down because letting them out in public is extremely dangerous. And what do we call these sorts: monsters, demons, animals...i.e., words for other sentient beings that don’t respect human worth. So if there are smart humans with values incompatible with human worth, then nothing prevents there from being AI with such values.
Andrew Burns
His take is so horrible. How could you be on the alien’s side? What argument could they make? It is jarring that even toying with the idea of causing the death of a particular group of people is strictly taboo and grounds for cancellation, but stating that you would seriously entertain arguments in favor of letting everyone die is okay. As if the biggest wrong is the discrimination rather than the death. Death is the biggest wrong and supporting omnicide is the worst possible position.
Upon reflection, he doesn’t say he is in favor of an alien plan where everyone gets killed. It could be that the alien AI offers something beneficial to humanity. However, “war” suggests that the interaction is not peaceful. I retract my claim that he is endorsing entertaining proposals for omnicide and substitute it with an observation that he is endorsing entertaining proposals from aliens which might include omnicidal or disempowering plans, and that he is not partial to humanity’s cause.
Yes. I agree with you. Innocence is naivete. They are the same thing. But innocence emphasizes the benefits of being unsullied by knowledge and naivete emphasizes the dangers.
Knowledge isn’t always psychically refreshing. I’ve heard people use the term cognitohazard to describe knowledge that causes mental harm to the person who knows it. Knowing about the wicked tendencies of man and the indifference of the universe is psychically scarring. Once you know about it, it alters your thought processes pretty permanently. It makes life sadder and fills it with more anxiety. However, because knowing about these things means that you can watch out for them and survive, not knowing can be dangerous.
In situations where not knowing does not present immediate harm, we use the word innocence. “Look, she is so friendly with everyone, even people she doesn’t know. Isn’t that precious. I wish I were still like that.” But in situations where not knowing places someone in danger, we use the word naivete. “Can you believe she gave that strange man all that information? She is so naive. He could come to her house and hurt her.”
As for innocence and naivete being associated with sexuality, the same reasoning holds. Sex, past and present, is dangerous business. In the past, getting pregnant meant you were at elevated risk of death or disability. Even if not impregnated, you could get incurable diseases that would shorten your life and make you unmarriagable. Lacking knowledge about sex meant you weren’t aware of this grim reality. And people who were aware of it wished they could go back to not knowing because the burden of knowledge is heavy. So they would say: “look, she is so innocent. Wish I were still so.” On the other hand, when people aware of the grim reality saw an innocent person acting in a way that was likely to attract unwanted sexual attention, they would call them naive, since this attention could lead to disease or pregnancy and therefore the discovery of the grim reality.
Exactly right. However, I am extremely doubtful about anyone who claims that all their patients are cured within a few sessions. That sounds very unlikely unless they screen out people with anything more than minor hang-ups. Sure, in many cases, the root cause of the psychological problem can be identified and the patient can learn a few techniques and then they no longer need further therapy. However, lots of people in therapy are dealing with negative mental processes that were baked into them by a difficult childhood or a traumatic experience. Those sorts of issues can require on-going therapy to keep the patient on track and in a positive mindspace. One quick tricks don’t work on someone with severe codependency or agoraphobia or anorexia. Maybe, with time, they can work through these issues and no longer need therapy, but this could take years.
Yes. This could have been written better. I am honestly, genuinely not partial to either side. I could be convinced that intelligence begets human-like values if someone could walk me through how that happens and how to account for examples of very intelligent people who do not have human-friendly values. I shouldn’t have been so aggressive with my word choice, I just find it frustrating when I see people assuming something will happen without explaining how. I reckon the belief in this convergence, at least sometimes, is tied to a belief in moral realism and that’s the crux. If moral realism holds, then moral truths are discoverable and, given sufficient intelligence, an entity will reach these truths by reason. For those who are not motivated by moral realism, then perhaps this conviction stems from sureness that the manner in which current AIs are trained—using human-created data—will, at some point, cause sufficiently advanced AI to grok human values. But what if they do grok, but don’t care? I understand that some animals eat their young, but I don’t endorse this behavior and wish it were not so. I would feel the urge to stop it if I saw it happening and would only refrain because I don’t want to interfere with natural processes. It seems to me that the world is not compatible with human values, otherwise what is would be what ought to be, so humans, which operate in the world, may not be compatible with AI values, even if the AI understands them deeply and completely.
Anyway, point is, I’m not trying to be a partisan.
I think this disagreement stems from a failure to distinguish which meaning of innocence we are talking about. By my reckoning, there are three major meanings: legal innocence, moral innocence, and naive innocence. Legal innocence is the lack of criminal culpability. Moral innocence is the lack of moral culpability. Naive innocence is the lack of knowledge about sensitive topics.
“Innocent as a dove and shrewd as a serpent” is referring to moral innocence and means: be clever, but only so far as is morally acceptable. Naive innocence, however, which is the topic the OP seems to be discussing, isn’t a virtue, it is ignorance, and curiosity is the virtue which seeks to extinguish it. An innocent listener who doesn’t understand the racial joke should be curious and ask probing questions and do research to better understand what the racial joke teller was trying to say. Then, the next time someone talks in a similar manner, the now savvy listener can make an informed decision about whether that person is the sort of person the listener wants to associate with.
First, I suggest that people pay heed to what happened in the movie “Don’t Look Up.” I don’t remember the character names, but the punk female scientist, when confronted during an interview by unserious journalists, went absolutely bonkers on television and contributed significantly to doom. The lesson I got from this is if you do not present serious existential threats in a cogent, sober manner, the public will polarize based on vibes and priors and then lock in. Only the best, most unflappable, polished spokespeople should be put forward, and even then, it might be no use.
Second, you cannot have a meaningful exchange on Twitter. Twitter encourages the generation of poorly reasoned emotional responses that are then used to undermine better reasoned future arguments. I would recommend people just avoid that platform entirely because the temptation to respond to raconteurs like Jezos is too high.
Sam Altman: is there a word for feeling nostalgic for the time period you’re living through at the time you’re living it?
Call it “nowstalgia.”
When I was in middle school, our instructor was trying to teach us about the Bill of Rights. She handed out a paper copy and I immediately identified that Article the first (sic) and Article the second (sic) were not among the first ten amendments and that the numbers for the others were wrong. I boldly asserted that this wasn’t the Bill of Rights and the teacher apologized and cursed the unreliable Internet. But I was wrong. This WAS the Bill of Rights, but the BILL rather than the ten ratified amendments. Everyone came away wrongly informed from that exchange.
Edit: I wrote before that I identified that they were not in the Constitution, but article the second is, as the 27th amendment, and I knew that, but it wasn’t among the first ten.
- [deleted]
I don’t know that an opinion that conforms to reality is self-reinforcing. It is reinforced by reality. The presence of a building on a map is reinforced by the continued existence of the building in real life.
“To believe” in German is glauben, also from Proto-Germanic. Was this meaning also colored by Greek?
Lex Luthor vibes.
This and other communities seek to transcend, or at least mitigate, human imperfections. Just because something is “human” doesn’t mean it contributes to human flourishing. Envy, rage, hate, and cruelty are human, after all.
Whenever someone uses “they,” I get nervous.
I’m puzzled by your use of the word “intelligence.” Intelligence refers to a capacity to understand facts, acquire knowledge and process information. Humans are presently the only members of the set of intelligent self-regulating systems.
People who eat seafood, but not the flesh of other terrestrial animals are pescatarian. Ethical (as opposed to environmental) pescatarians say fish and other marine life aren’t complex enough for fear or pain. Perhaps they call themselves vegetarians just to avoid having to explain pescatarianism.
Most of the time English has an antonym that does not involve a negative prefix or suffix.
It is not warm. ~= It is cool.
It is not new. ~= It is old.
But this is not the case in other languages. Consider Esperanto:
It is not warm. → Ĝi ne estas varmeta. ~= Ĝi estas malvarmeta.
It is not new. → Ĝi ne estas nova. ~= Ĝi estas malnova.
Because mal- is equivalent to un-, it is forbidden, and you have to resort to periphrasis:
Ĝi estas alia ol varmeta. (It is other than warm.)
Ĝi estas la malo de varmeta. (It is the opposite of warm.)...oh, wait, this contains mal- too.
Lord Acton was on to something when he observed that great men become bad men when given great power. “Despotic power is always accompanied by corruption of morality.” I believe this is because morality flows from the knowledge of our own fragility...there but for the grace of God go I, the faithful might say...and the golden rule works because people stand on an equal playing field and recognize that bad actions have a tendency to boomerang and that anyone could be caught in a state of misfortune. So, when we reflect as we are now, weak mortals, we converge on a certain set of values. But, when given enormous powers, it is very likely we will converge on different values, values which might cause our present selves to shriek in disgust. No one can be trusted with absolute power.