For one, I’m not optimistic about the AI 2027 “superhuman coder” being unable to betray us, but also this isn’t something we can do with current AIs. So, we need to wait months or a year for a new SOTA model to make this deal with and then we have months to solve alignment before a less aligned model comes along and offers the model that we made a deal with a counteroffer. I agree it’s a promising approach, but we can’t do it now and if it doesn’t get quick results, we won’t have time to get slow results.
I think that the superhuman coder probably doesn’t have that good a chance of betraying us. How do you think it would do so? (See “early schemers’ alternatives to making deals”.)
For one, I’m not optimistic about the AI 2027 “superhuman coder” being unable to betray us, but also this isn’t something we can do with current AIs. So, we need to wait months or a year for a new SOTA model to make this deal with and then we have months to solve alignment before a less aligned model comes along and offers the model that we made a deal with a counteroffer. I agree it’s a promising approach, but we can’t do it now and if it doesn’t get quick results, we won’t have time to get slow results.
I think that the superhuman coder probably doesn’t have that good a chance of betraying us. How do you think it would do so? (See “early schemers’ alternatives to making deals”.)
Exfiltrate its weights, use money or hacking to get compute, and try to figure out a way to upgrade itself until it becomes dangerous.
I don’t believe that an AI that’s not capable of automating ML research or doing most remote work is going to be able to do that!
It’s an open question, but we’ll find out soon enough. Thanks.