Thou shalt not command an alighned AI

Raymond is tired. He exhails exhaustly: >>I don’t think we even know what alighnment is, like we are not able to define it.<<

I hop up on my chair in the meditarian restaurant: >I disagree, if you give me 3 seconds, I can define it.<

>>---<<

>Can we narrow it to alighnment of AI to humans?<

>>Yes, let’s narrow it to alighnment of one AI to one person.<<

>Fine. The AI is alighned if you give it a goal and it follows towards that goal without modifying it with its own intentions or goals.<

>>That sounds a bit way too abstract...<<

>Yeah, but in what sense, you mean?<

>>Like the goal, what is that, more precisely?<<

>That is a state of the world you want to achieve or a series of states of the world.<

>>Oh, but how would you specify that?<<

>You can specify it, describe it, in infinetly many ways, there is a scale of how detailed description you choose, which will imply a level of approximation of the state.<

>>Oh, but that won’t describe the state completely..?<<

>Well maybe if you can describe to the quantum state level, but surely that is not practical.<

>>So then the AI must somehow interpret your goal, right?<<

>Ehmmm, well no, but what you mean it would have to interpolate to fill in the under-specified spots in the description of your goal..?<

>>Yes, that is a good expression for what would need to happen.<<

>Then what we’ve discovered here is another axis, orthogonal to alighnment, which would control to what level of under-specifiedness we want the AI to interpolate and where it would need to ask you to fill in the gaps (more) before moving towards your goal.<

>>Oh, but we also can’t be like “Create a picture of a dog” and then ’d need to specify each pixel.<<

>Sure. But maybe the AI must ask you whether you want the picture on paper or digitally on your screen, with a reasonable threshold for clarification.<

>>Hmm, but people want things they do not have...<<

>and they can end up in a state they feel bad in with an alighned AI.<

>>So what do you do to make the alighment guarantee good outcomes? People are stupid...<<

>and that’s on them. You can call it incomptetence, but I’d call that a misuse.<