Those last two words are doing a supermajority of the work!
And yes, it’s about uneven distribution of power—but that power gradient can shift towards ASI pretty quickly, which is the argument. Still, the normative concern that most humans lost control already stands.
The president would probably be part of the supermajority and therefore cooperative, and it might work even if they aren’t.
We’re seeing this fail in certain places in real time today in the US. But regardless, the assumption of correlation of preferences often fails, partly due to the power imbalances themselves.
Those last two words are doing a supermajority of the work!
And yes, it’s about uneven distribution of power—but that power gradient can shift towards ASI pretty quickly, which is the argument. Still, the normative concern that most humans lost control already stands.
We’re seeing this fail in certain places in real time today in the US. But regardless, the assumption of correlation of preferences often fails, partly due to the power imbalances themselves.