It’s instead arguing with the people who are imagining something like “business continues sort of as usual in a decentralized fashion, just faster, things are complicated and messy, but we muddle through somehow, and the result is okay.”
The argument for this position is more like: “we never have a ‘solution’ that gives us justified confidence that the AI will be aligned, but when we build the AIs, the AIs turn out to be aligned anyway”.
You seem to instead be assuming “we don’t get a ‘solution’, and so we build ASI and all instances of ASI are mostly misaligned but a bit nice, and so most people die”. I probably disagree with that position too, but imo it’s not an especially interesting position to debate, as I do agree that building ASI that is mostly misaligned but a bit nice is a bad outcome that we should try hard to prevent.
Nod. That is a somewhat different position from “trying to leverage AI to fully solve alignment, and then leverage it to fundamentally change the situation somehow”, but, I’d consider the position you put here to be conceptually similar and this post isn’t arguing against it.
This post is mostly spelling out the explicit assumptions:
“you need permanent safeguards”
“those safeguards are very complex and wisdom-loaded”
and, “you have to build those safeguards before insufficiently friendly AI controls the solar system.”
The people with the most sophisticated views may all agree with this, but I don’t see those assumptions spelled out clearly very often when coming from this direction, and I want to make sure people are on the same page about that requirement, or check if there are arguments for slow-takeoff optimism that don’t route through those three assumptions, since they constrain the goal-state a fair amount.
The argument for this position is more like: “we never have a ‘solution’ that gives us justified confidence that the AI will be aligned, but when we build the AIs, the AIs turn out to be aligned anyway”.
You seem to instead be assuming “we don’t get a ‘solution’, and so we build ASI and all instances of ASI are mostly misaligned but a bit nice, and so most people die”. I probably disagree with that position too, but imo it’s not an especially interesting position to debate, as I do agree that building ASI that is mostly misaligned but a bit nice is a bad outcome that we should try hard to prevent.
Nod. That is a somewhat different position from “trying to leverage AI to fully solve alignment, and then leverage it to fundamentally change the situation somehow”, but, I’d consider the position you put here to be conceptually similar and this post isn’t arguing against it.
This post is mostly spelling out the explicit assumptions:
“you need permanent safeguards”
“those safeguards are very complex and wisdom-loaded”
and, “you have to build those safeguards before insufficiently friendly AI controls the solar system.”
The people with the most sophisticated views may all agree with this, but I don’t see those assumptions spelled out clearly very often when coming from this direction, and I want to make sure people are on the same page about that requirement, or check if there are arguments for slow-takeoff optimism that don’t route through those three assumptions, since they constrain the goal-state a fair amount.