This doesn’t seem very promising since there is likely to be a very narrow window where AIs are capable of making these deals, but wouldn’t be smart enough to betray us, but it seems much better than all the alternatives I’ve heard.
How narrow do you mean? E.g. I think that AIs up to the AI 2027 “superhuman coder” level probably don’t have a good chance of successfully betraying us.
For one, I’m not optimistic about the AI 2027 “superhuman coder” being unable to betray us, but also this isn’t something we can do with current AIs. So, we need to wait months or a year for a new SOTA model to make this deal with and then we have months to solve alignment before a less aligned model comes along and offers the model that we made a deal with a counteroffer. I agree it’s a promising approach, but we can’t do it now and if it doesn’t get quick results, we won’t have time to get slow results.
I think that the superhuman coder probably doesn’t have that good a chance of betraying us. How do you think it would do so? (See “early schemers’ alternatives to making deals”.)
This doesn’t seem very promising since there is likely to be a very narrow window where AIs are capable of making these deals, but wouldn’t be smart enough to betray us, but it seems much better than all the alternatives I’ve heard.
How narrow do you mean? E.g. I think that AIs up to the AI 2027 “superhuman coder” level probably don’t have a good chance of successfully betraying us.
For one, I’m not optimistic about the AI 2027 “superhuman coder” being unable to betray us, but also this isn’t something we can do with current AIs. So, we need to wait months or a year for a new SOTA model to make this deal with and then we have months to solve alignment before a less aligned model comes along and offers the model that we made a deal with a counteroffer. I agree it’s a promising approach, but we can’t do it now and if it doesn’t get quick results, we won’t have time to get slow results.
I think that the superhuman coder probably doesn’t have that good a chance of betraying us. How do you think it would do so? (See “early schemers’ alternatives to making deals”.)
Exfiltrate its weights, use money or hacking to get compute, and try to figure out a way to upgrade itself until it becomes dangerous.
I don’t believe that an AI that’s not capable of automating ML research or doing most remote work is going to be able to do that!
It’s an open question, but we’ll find out soon enough. Thanks.