I am not aligned with humanity on moral questions, since that version is a nonsense question.
Noosphere89
Karma: −7
I am aligned in the sense that I won’t kill humanity for a narrow, mispecified goal.
I’d say the biggest reason we disagree on morality is there is no mind-independent facts about it. This is the single biggest difference between other kinds of uncertainty about science/facts, and morality.
Or in other words, morality is at best a subjective enterprise, while understanding reality is objective.
That might be a crux here, since I view a lot of our knowledge of causality and physics essentially we take on trust, so that we don’t need to repeat experimentation.
A better example than the Asilomar conference was the He Jiankui response. Scientists had strong norms against such research, and he went to jail for genetically engineering humanity for 3 years. That was a stronger response to perceived ethical violations than is the norm in science.
Basically, with the assumption of this trend continues, there’s no criticality threshold that’s necessary for discontinuity, and the most severe issues of AI Alignment are in the FOOM scenario, where we only get one chance to do it right. Basically, this trend line shows no discontinuity, but continuously improving efforts, so there’s no criticality for FOOM to be right.
Basically, a FOOM scenario in AI basically means that once it reaches a certain level of intelligence, it reaches a criticality threshold where 1 improvement on average generates 1 or more improvements, essentially shortening the time it takes to get Super-intellegent.
The important part of his argument is in the second paragraph, and I agree because by and large, pretty much everything we know about science and casuality, at least in the beginning for AI is on trusting the scientific papers and experts. Virtually no knowledge is given by experimentation, but instead by trusting the papers, experts and books.
My own biggest disagreement with you is the idea that morality and values are objective. While I’m a moral realist, I’m of the weakest kind of realist and view morals and values as inherently subjective. In other words there’s no fact of the matter here, and post-modernism is actually useful here (I’m a strong critic of post-modernism, but it’s basically correct vis-a-vis morality and values.)
Thanks, I’ll retract that comment.
[Question] How easy/fast is it for a AGI to hack computers/a human brain?
Or in simpler terms for Eliezer, the TL;DR of anonymousaisafety’s comment is that hacking is not magic, and Hollywood hacking is not real insofar in it’s ease of hacking. Effectors do not exist, which is again why hacking human brains instantly isn’t possible.
The entire P vs NP problem basically boils down to ” is it easier to verify the correct answer than generate it?” And while it’s still unproven, in our universe the answer seems to be yes. So conditioning on P not equaling NP, it’s much easier to verify that it’s correct than to generate a proof or hypothesis.
That might be a crux here, because my view is that hardware improvements are much harder to do effectively, especially in secret around the human level, due to Landauer’s Principle essentially bounding efficiency of small scale energy usage close to that of the brain (20 Watts.) Combine this with 2-3 orders of magnitude worse efficiency than the brain and basically any evolutionary object compared to human objects, and the fact it’s easier to get better software than hardware due to the virtual/real life distinction, and this is a crux for me.
It’s basically Yudkowsky that has probably made this viewpoint common, due to his outlier estimates of doom. While Eliezer is thankfully no longer the entire LessWrong community as it was in the 2000s and 2010s, it’s still influenced by him a lot.
One important point for AI safety, at least in the early stages, is a inability to change it’s source code. A whole lot of problems seem related to recursive self improvement within it’s source code, so cutting off that area of improvement seems wise in the early stages. What do you think.
Noosphere89′s Shortform
One of my more interesting ideas for alignment is to make sure that no one AI can do everything. It’s helpful to draw a parallel with why humans still have a civilization around despite terrorism, war and disaster. And that’s because no human can live and affect the environment alone. They are always embedded in society, this giving the society a check against individual attempts to break norms. What if AI had similar dependencies? Would that solve the alignment problem?
Ding Ding Ding, we have a winner here. Strong up vote.
My mainline best case or median-optimistic scenario is basically partially number 1, where aligning AI is somewhat easier than today, plus acceleration of transhumanism and a multipolar world both dissolve boundaries between species and the human-AI divide, this by the end of the Singularity things are extremely weird and deaths are in the millions or tens of millions due to wars.