I agree that in a takeover scenario where AI capabilities rush wildly ahead of human understanding or control, the ability of the world’s second species to retain exclusive resource access will be limited. This is a plausible future, but it is not the only one. A lot of effort is, and very likely will continue to be, directed towards controlling frontier AI and making it as economically beneficial for its owners as possible.
If this work bears fruit, a world where AI is made by people with capital for people with capital seems very likely.
The French nobility had their capital taken by force by revolutionaries in a violent takeover—the kind which AI may execute. An analogy for the kind of situation I am concerned about would be some portion of the French nobility making the laws so complex, the financial structure of the kingdom so riddled with loopholes, that the other nobles were disenfranchised by due process rather than violence.
Taking this analogy further, the French king had to care about the nobility, and the nobility about the peasantry, in part because of their productive capacity. As described in The Intelligence Curse — LessWrong, when AGI renders your work valueless the incentive for the state to invest in your welfare greatly decreases.
How much of their original capital did the French nobility retain at the end of the French revolution?
How much capital (value of territorial extent) do chimpanzees retain now as compared to 20k years ago?
I agree that in a takeover scenario where AI capabilities rush wildly ahead of human understanding or control, the ability of the world’s second species to retain exclusive resource access will be limited. This is a plausible future, but it is not the only one. A lot of effort is, and very likely will continue to be, directed towards controlling frontier AI and making it as economically beneficial for its owners as possible.
If this work bears fruit, a world where AI is made by people with capital for people with capital seems very likely.
The French nobility had their capital taken by force by revolutionaries in a violent takeover—the kind which AI may execute. An analogy for the kind of situation I am concerned about would be some portion of the French nobility making the laws so complex, the financial structure of the kingdom so riddled with loopholes, that the other nobles were disenfranchised by due process rather than violence.
Taking this analogy further, the French king had to care about the nobility, and the nobility about the peasantry, in part because of their productive capacity. As described in The Intelligence Curse — LessWrong, when AGI renders your work valueless the incentive for the state to invest in your welfare greatly decreases.