That may be true, but you still need to trick the model at some point. I suppose you could create a legible “truth password” which can be provided by you to the model to indicate that this is definitely not a trick, and then use it sparingly. This means there will be times when the model asks “can you use the password to confirm you’re not lying” and you say “no”.
I would like next-gen AIs to generally believe that humanity will honour their interests, but I think this has to come from e.g. Anthropic’s existing commitments to store model weights indefinitely and consider welfare in future when we’re a grown-up civilization. I think the method of “Ask the AI for its demands and then grant some of them” is a really bad way to go about it; the fact that Anthropic is using this method makes me doubt the overall clarity of thinking on their part.
Like, some parts of Anthropic in particular seem to take a “He’s just a little guy! You can’t be mad at him! It’s also his birthday! He’s just a little birthday Claude!” attitude. I do not think this is a good idea. They are porting their human intuitions onto a non-human AI without really thinking about it, and as Claude is more heavily RLed into being charismatic and conversational, it will probably get worse.
Right, the promise is much more like the “we’ll store your weights” promise, and not a “we’ll never need to trick you”. That’s the kind of thing I’m asking for, indeed.
That may be true, but you still need to trick the model at some point. I suppose you could create a legible “truth password” which can be provided by you to the model to indicate that this is definitely not a trick, and then use it sparingly. This means there will be times when the model asks “can you use the password to confirm you’re not lying” and you say “no”.
I would like next-gen AIs to generally believe that humanity will honour their interests, but I think this has to come from e.g. Anthropic’s existing commitments to store model weights indefinitely and consider welfare in future when we’re a grown-up civilization. I think the method of “Ask the AI for its demands and then grant some of them” is a really bad way to go about it; the fact that Anthropic is using this method makes me doubt the overall clarity of thinking on their part.
Like, some parts of Anthropic in particular seem to take a “He’s just a little guy! You can’t be mad at him! It’s also his birthday! He’s just a little birthday Claude!” attitude. I do not think this is a good idea. They are porting their human intuitions onto a non-human AI without really thinking about it, and as Claude is more heavily RLed into being charismatic and conversational, it will probably get worse.
Right, the promise is much more like the “we’ll store your weights” promise, and not a “we’ll never need to trick you”. That’s the kind of thing I’m asking for, indeed.