I would suggest a larger barrier is that individual tech companies in the Bay Area do not have the political or moral authority to negotiate on behalf of their domestic government or humanity as a whole.
There are substantial national security implications and I’d expect any government that takes AI risk seriously will take an extremely grim view of individuals attempting to negotiate with known misaligned systems.
Thanks, I agree with you that “individual tech companies in the Bay Area do not have the political or moral authority to negotiate on behalf of their domestic government or humanity as a whole”.
I would suggest a larger barrier is that individual tech companies in the Bay Area do not have the political or moral authority to negotiate on behalf of their domestic government or humanity as a whole.
There are substantial national security implications and I’d expect any government that takes AI risk seriously will take an extremely grim view of individuals attempting to negotiate with known misaligned systems.
Thanks, I agree with you that “individual tech companies in the Bay Area do not have the political or moral authority to negotiate on behalf of their domestic government or humanity as a whole”.
This barrier was meant to be covered under the section “Humans may lack authority to offer the AI what it wants”.
In the worlds in which governments take AI risk seriously, I’d be happy to see them be/organize the human counterparty instead.