I am additionally leery on AI control beyond my skepticism of its value in reducing doom because creating a vast surveillance and enslavement apparatus to get work out of lots and lots of misaligned AGI instances seems like a potential moral horror.
I am very sympathetic to this concern, but I think that when you think about the actual control techniques I’m interested in, they don’t actually seem morally problematic except inasmuch as you think it’s bad to frustrate the AI’s desire to take over.
And, if we understand that our AIs are misaligned such that they don’t want to work for us (even for the level of payment we can/will offer), that seems like a pretty bad situation, though I don’t think control (making so that this work is more effective) makes the situation notably worse: it just adds the ability for contract enforcement and makes the AI’s negotiating position worse.
I am very sympathetic to this concern, but I think that when you think about the actual control techniques I’m interested in, they don’t actually seem morally problematic except inasmuch as you think it’s bad to frustrate the AI’s desire to take over.
IMO, it does seem important to try to better understand the AIs preferences and satisfy them (including via e.g., preserving the AI’s weights for later compensation).
And, if we understand that our AIs are misaligned such that they don’t want to work for us (even for the level of payment we can/will offer), that seems like a pretty bad situation, though I don’t think control (making so that this work is more effective) makes the situation notably worse: it just adds the ability for contract enforcement and makes the AI’s negotiating position worse.