...and I think this is the positive update. It feels very plausible, in a visceral way, that the first economically transformative AI systems could be, in many ways, really dumb.
I think this is right, but I think is misleadingly encouraging.
I keep having to remind myself: Most of the risk does not come from the early transformative AIs. Most of the risk comes from the overwhelming superintelligences that come only a few years later.
Maybe we can leverage our merely transformative capabilities into a way to stick the landing with the overwhelming superintelligences, but that’s definitely not a forgone conclusion.
yeah I agree, I think the update is basically just “AI control and automated alignment research seem very viable and important”, not “Alignment will be solved by default”
I think this is right, but I think is misleadingly encouraging.
I keep having to remind myself: Most of the risk does not come from the early transformative AIs. Most of the risk comes from the overwhelming superintelligences that come only a few years later.
Maybe we can leverage our merely transformative capabilities into a way to stick the landing with the overwhelming superintelligences, but that’s definitely not a forgone conclusion.
yeah I agree, I think the update is basically just “AI control and automated alignment research seem very viable and important”, not “Alignment will be solved by default”