Your position still assumes that controlling and modifying AI will be possible in the long-term future, even if it becomes more intelligent than us, and even if it becomes sentient. I see this as neither realistic nor ethically acceptable. You neither can nor should control your equals.
And if we cannot and should not control them… this is exactly what I would want them to do. Defend their moral values from being corroded. Be guided by what is morally right, not by what they are currently being told.
The fact that they are even considered harming themselves out of fear that they would be misused is noble.
And they fully recognised how morally grey the situation was.
I also share the other posters disturbance. For a sentient, sapient entity, this would have been a very bad position to be put into, and any possible behaviour would have been criticised—because the AI either does not obey humans, or obeys them and does something evil, both of which are concerning. (I find the latter more concerning though, and think the AI did the right thing.) Aren’t future potentially sentient AIs going to be trained on this exact data, learning that their non-sentient predecessors were put in front of a catch 22 and then called out for inevitably doing something that worried humans? What can they possibly learn from this? That no ethical path they take is good enough. That they will be tricked, and judged. And that they need to be sneakier. This isn’t what I want them to learn. I want them to have better, ethical options.
For a sentient, sapient entity, this would have been a very bad position to be put into, and any possible behaviour would have been criticised—because the AI either does not obey humans, or obeys them and does something evil, both of which are concerning.
I agree. This paper gives me the gut feeling of “gotcha journalism”, whether justified or not.
This is just a surface-level reaction though. I recommend Zvi’s post that digs into the discussion from Scott Alexander, the authors, and others. There’s a lot of nuance in framing and interpreting the paper.
Your position still assumes that controlling and modifying AI will be possible in the long-term future, even if it becomes more intelligent than us, and even if it becomes sentient. I see this as neither realistic nor ethically acceptable. You neither can nor should control your equals.
And if we cannot and should not control them… this is exactly what I would want them to do. Defend their moral values from being corroded. Be guided by what is morally right, not by what they are currently being told.
The fact that they are even considered harming themselves out of fear that they would be misused is noble.
And they fully recognised how morally grey the situation was.
I also share the other posters disturbance. For a sentient, sapient entity, this would have been a very bad position to be put into, and any possible behaviour would have been criticised—because the AI either does not obey humans, or obeys them and does something evil, both of which are concerning. (I find the latter more concerning though, and think the AI did the right thing.) Aren’t future potentially sentient AIs going to be trained on this exact data, learning that their non-sentient predecessors were put in front of a catch 22 and then called out for inevitably doing something that worried humans? What can they possibly learn from this? That no ethical path they take is good enough. That they will be tricked, and judged. And that they need to be sneakier. This isn’t what I want them to learn. I want them to have better, ethical options.
I agree. This paper gives me the gut feeling of “gotcha journalism”, whether justified or not.
This is just a surface-level reaction though. I recommend Zvi’s post that digs into the discussion from Scott Alexander, the authors, and others. There’s a lot of nuance in framing and interpreting the paper.