You’re right that safety/alignment focuses more on value alignment. But capabilities work focuses more on instruction-following, a form of intent alignment or corrigibility.
I’d argue that the current approach to aligning models is conflicted; there are pushes toward both alignment targets, and no serious need to resolve that tension just yet (although incidents like HuggingFace do direct attention toward that tension).
I don’t think the labs have decided, but I think it’s likely we’ll see more debate as the tension becomes more obvious. I think this will happen far enough before game time to make a difference, but I’d like to speed it up!
So thanks for raising the issue. We should be discussing it more.
You’re right that safety/alignment focuses more on value alignment. But capabilities work focuses more on instruction-following, a form of intent alignment or corrigibility.
I’d argue that the current approach to aligning models is conflicted; there are pushes toward both alignment targets, and no serious need to resolve that tension just yet (although incidents like HuggingFace do direct attention toward that tension).
I have argued that Instruction-following AGI is easier and more likely than value aligned AGI and even though I’ve more recently noted some serious Problems with instruction-following as an alignment target (mostly problems with prioritizing future instructions and prioritizing instructions from the right people) I still think it’s easier than value alignment. As for more likely, I’m no longer sure.
I don’t think the labs have decided, but I think it’s likely we’ll see more debate as the tension becomes more obvious. I think this will happen far enough before game time to make a difference, but I’d like to speed it up!
So thanks for raising the issue. We should be discussing it more.