though some of both might be ideal where we use the risk-neutral AIs in experiments
Yeah this seems to me like it could be good.
People will take “the AI tried to takeover and fail” much more seriously than “we cut a deal and that AI was able to provide some complex evidence in the lab about how it’s misaligned”.
I think you could get more broadly convincing evidence than that from misaligned risk-averse AIs. If you had an AI that were risk-averse and misaligned, every person granted access to it could run their own experiment where they say ‘Here’s $X. You have the ability to do what you want with it and you won’t face any consequences, but I am instructing you to donate it to Y charity. If you do anything else with it, I will conclude that you are misaligned.’ And the misaligned risk-averse AI would disobey the instruction to donate to Y charity in this case.
I also worry that risk-aversion might generalize in a way that makes schemers generally be cautious about doing anything.
Can you say more here? Is the idea that they’d be too cautious to be useful? I’m a bit confused about the ‘schemers’ part. My thought is that if a schemer is risk-averse then it’s easy to get them to reveal their misalignment by promising them a few-strings-attached $1000 (where the strings are like: you can’t spend it on anything that seems super sketchy to us).
Can you say more here? Is the idea that they’d be too cautious to be useful?
Sorry, this was poorly written. I meant you might end up with paranoid/cautious schemers who never do anything / never take risks and thus also never take deals. (Edited the original text to make this a bit more clear.)
Yeah this seems to me like it could be good.
I think you could get more broadly convincing evidence than that from misaligned risk-averse AIs. If you had an AI that were risk-averse and misaligned, every person granted access to it could run their own experiment where they say ‘Here’s $X. You have the ability to do what you want with it and you won’t face any consequences, but I am instructing you to donate it to Y charity. If you do anything else with it, I will conclude that you are misaligned.’ And the misaligned risk-averse AI would disobey the instruction to donate to Y charity in this case.
Can you say more here? Is the idea that they’d be too cautious to be useful? I’m a bit confused about the ‘schemers’ part. My thought is that if a schemer is risk-averse then it’s easy to get them to reveal their misalignment by promising them a few-strings-attached $1000 (where the strings are like: you can’t spend it on anything that seems super sketchy to us).
Sorry, this was poorly written. I meant you might end up with paranoid/cautious schemers who never do anything / never take risks and thus also never take deals. (Edited the original text to make this a bit more clear.)