I confess to not reading this long article before commenting.
If you’re worried about powerful AIs, why not assume they’d figure this stuff out themselves? And how would testing current AIs be relevant to the bargaining behavior of the powerful AIs you’re worried about?
This whole research line just really feels like it’s in the weeds. There aren’t enough people working on conceptual alignment to explore all of the interesting-and-potentially-useful avenues like this, so I really wish you were working on something more recognizeably likely impactful. Or perhaps just making a more prominent and clearer argument for how this is likely to be impactful.
There’s more explanation of why work on AI conflict is time-sensitive / not purely deferrable to AIs in this sequence, which I link to in the intro. And as I note in the intro:
We agree SPIs will likely be used by default. However, this is arguably not overwhelmingly likely, because AIs or humans in the loop might mistakenly lock out the opportunity to use SPIs later. It’s unclear if default capabilities progress will generalize to careful reasoning about novel bargaining approaches. So, given the large stakes of conflicts that SPIs could prevent, making SPI implementation even more likely seems promising overall.
To be clear, I don’t expect this to be super compelling on its own. I’m assuming readers have background on CLR’s agenda more generally. And this agenda is meant to motivate and explain our strategy on an approach that’s particularly promising if you already share CLR’s general prioritization.
Re: this:
And how would testing current AIs be relevant to the bargaining behavior of the powerful AIs you’re worried about?
My sense is that it’s fairly common in AI safety generally to study current AIs in preparation for the actually dangerous AIs. I don’t understand why you think this methodology is especially ill-suited for our threat model.
I confess to not reading this long article before commenting.
If you’re worried about powerful AIs, why not assume they’d figure this stuff out themselves? And how would testing current AIs be relevant to the bargaining behavior of the powerful AIs you’re worried about?
This whole research line just really feels like it’s in the weeds. There aren’t enough people working on conceptual alignment to explore all of the interesting-and-potentially-useful avenues like this, so I really wish you were working on something more recognizeably likely impactful. Or perhaps just making a more prominent and clearer argument for how this is likely to be impactful.
There’s more explanation of why work on AI conflict is time-sensitive / not purely deferrable to AIs in this sequence, which I link to in the intro. And as I note in the intro:
To be clear, I don’t expect this to be super compelling on its own. I’m assuming readers have background on CLR’s agenda more generally. And this agenda is meant to motivate and explain our strategy on an approach that’s particularly promising if you already share CLR’s general prioritization.
Re: this:
My sense is that it’s fairly common in AI safety generally to study current AIs in preparation for the actually dangerous AIs. I don’t understand why you think this methodology is especially ill-suited for our threat model.