Nod. That is a somewhat different position from “trying to leverage AI to fully solve alignment, and then leverage it to fundamentally change the situation somehow”, but, I’d consider the position you put here to be conceptually similar and this post isn’t arguing against it.
This post is mostly spelling out the explicit assumptions:
“you need permanent safeguards”
“those safeguards are very complex and wisdom-loaded”
and, “you have to build those safeguards before insufficiently friendly AI controls the solar system.”
The people with the most sophisticated views may all agree with this, but I don’t see those assumptions spelled out clearly very often when coming from this direction, and I want to make sure people are on the same page about that requirement, or check if there are arguments for slow-takeoff optimism that don’t route through those three assumptions, since they constrain the goal-state a fair amount.
Nod. That is a somewhat different position from “trying to leverage AI to fully solve alignment, and then leverage it to fundamentally change the situation somehow”, but, I’d consider the position you put here to be conceptually similar and this post isn’t arguing against it.
This post is mostly spelling out the explicit assumptions:
“you need permanent safeguards”
“those safeguards are very complex and wisdom-loaded”
and, “you have to build those safeguards before insufficiently friendly AI controls the solar system.”
The people with the most sophisticated views may all agree with this, but I don’t see those assumptions spelled out clearly very often when coming from this direction, and I want to make sure people are on the same page about that requirement, or check if there are arguments for slow-takeoff optimism that don’t route through those three assumptions, since they constrain the goal-state a fair amount.