I must say, you have a very pessimistic/optimistic view of AI would be able to solve P=NP. I won’t say you’re completely wrong, as there’s always a chance that P does equal NP. But I would be very careful of predicting anything based on the possibility of P=NP.
Noosphere89
Because of it’s low chance of existential risk or a singularity utopia. Here’s the thing, technologies are adopted first at a low level and at early adopters, then it becomes cheaper and better, than it more or less becomes very popular. No technology ever had the asymptotic growth or singularity that ML/AI advocates claim to have happened. So we should be very skeptical about any claims of existential risks.
On climate change, we both know it will be serious and that it is not an existential risk or a civilization collapse disaster.
I will ask this question, is the Singularity/huge discontinuity scenario likely to happen? Because I see this as a meta-assumptionn behind all the doom scenarios, so we need to know whether the Singularity can happen and will happen.
This. Combine this fact with the non-trivial chance that moral values are subjective, not objective, and there is little good reason to be doing alignment.
Not really, I do want to make an AGI, primarily because I have very much the want to have a singularity, as it represents hope to me, and I have very different priors than Eliezer or MIRI about how much we’re doomed.
Notably, the case for certain doom as proposed by Rob Bensinger et al relies on 3 assumptions that needs to be tested: A: The Singularity/AI Foom scenarios are likely. I have my problems with this assumption, but I will accept it to show why that doesn’t lead to certain doom.
The next assumption is B: That AI will all have the same goals and that these goals all lead to destroying humanity. Now this is a case where I see factions forming on this question, and naively I expect a bell curve of opinions on this question, as well as many different opinions. I don’t expect coordination of all AIs to destroy humanity not because of fundamental incapability, but because I don’t expect unification of opinions here.
And finally, this rests on assumption C: that humanity and it’s descendants are narrowly defined. The nice thing about AI Foom scenarios is that while I don’t expect instant technology to come online, it also makes transhumanism far easier than otherwise, quickly closing the gap. That doesn’t mean it’s all sunny and rainbows, but we are spared certain doom by this.
Unfortunately, cryopreservation isn’t nearly as reliable as needed in order to assume immortality is achieved. While we’ve gotten better at it, it still relies on toxic chemicals in order to vitrify the brain.
I do see this as a blind spot, and perhaps may be giving this problem a harder task than what needs to happen.
I agree with the need for “skin in the game”, for most the same reasons as you, and I think the AI Alignment field is falling prey to the unilateralist’s curse here.
True, I’ve got to be more specific in my wording when I talk about stuff. And I’ll read that link you’ve gave me.
The answer is because the orthogonality thesis means there is no correlation between goals and intelligence level.
Some conclusions from this, assuming this holds for AGI and ASI are:
A: Eliezer was wrong about Foom being plausible, and that is probably the single most x-risk reduction for MIRI. Re AI: I’d update from a 90-99% chance of x-risk to a maximum of 30%, and often the risk estimate would be 1-10% percent due to uncertainties beyond the Singularity/Foom hypothesis.
B: We will get AGI, and it’s really a matter of time. This means we still must do safety in AI.
This is an ambiguous result for everyone in AI safety. On the one hand, we probably can get by with partial failures, primarily because there is no need to one-shot everything, so it probably won’t mean the end of human civilization. On the other hand, it does mean that we still have to do AI safety.
The only reason we care about AI Safety is because we believe the consequences are potentially existential. If it wasn’t, there would be no need for safety.
This. A lot of the blame goes to MIRI viewing AI Alignment discretely, rather than continuously, as well as a view that only heroic or pivotal acts save the world. This tends to be all or nothing, and generates all-or-nothing views.
Faster than biological brains, by 6 orders of magnitude.
Ding Ding Ding, we have a winner here. Strong up vote.
One of my more interesting ideas for alignment is to make sure that no one AI can do everything. It’s helpful to draw a parallel with why humans still have a civilization around despite terrorism, war and disaster. And that’s because no human can live and affect the environment alone. They are always embedded in society, this giving the society a check against individual attempts to break norms. What if AI had similar dependencies? Would that solve the alignment problem?
One important point for AI safety, at least in the early stages, is a inability to change it’s source code. A whole lot of problems seem related to recursive self improvement within it’s source code, so cutting off that area of improvement seems wise in the early stages. What do you think.
That might be a crux here, because my view is that hardware improvements are much harder to do effectively, especially in secret around the human level, due to Landauer’s Principle essentially bounding efficiency of small scale energy usage close to that of the brain (20 Watts.) Combine this with 2-3 orders of magnitude worse efficiency than the brain and basically any evolutionary object compared to human objects, and the fact it’s easier to get better software than hardware due to the virtual/real life distinction, and this is a crux for me.
Yes, I would do a modern democracy, because of one fact that is shown in stark relief by the Ukraine War: Democracy isn’t perfect, but is usually immune to the unilateralist’s curse and is better at governance than any other system we’ve tried.
To quote Churchill: “Democracy is the worst form of government, except for all the other systems we’ve tried.