What exactly is the “economic incentive” that keeps the capitalist in power in the modern world, given that all they have is a piece of paper saying that they “own” the factory or the farm? It seems like you could make an isomorphic argument for an inevitable proletarian revolution, and in fact I’d find it more intuitively persuasive than what you are saying here. But in fact it’s easy to have systems of power which are perpetuated despite being wildly out of line with the real physical importance of each faction.
(Perhaps your analogous story would be that capitalists with legal ownership are mostly disempowered in the modern world, and it’s managers and people with relevant expertise and understanding who inevitably end up with the power? I think there’s something to that, but nevertheless the capitalists do have a lot of formal control and it’s not obviously dwindling.)
I also don’t really think it’s clear that AGI means that capitalists are the only folks who matter in the state of anarchy. Instead it seems like their stuff would just get taken from them. In fact there just doesn’t seem to be any economic incentives at all of the kind of you seem to be gesturing at, any humans are just as economically productive as any others, so the entire game is the self-perpetuating system of power where people who call the shots at time T try to make sure that they keep calling the shots at time T+1. That’s a complicated dynamic and it’s not clear where it goes, but I’m skeptical about this methodology for confidently forecasting it.
And finally, this is all on top of the novel situation that democratic states are nominally responsible to their voters, and that AI makes it radically easier to translate this kind of de jure control into de facto control (by reducing scope for discretion by human agents and generally making it possible to build more robust institutions).
I think the perspective you are expressing here is quite common and I’m not fully understanding or grappling with it. I expect it would be a longer project for me to really understand it (or for you or someone else to really lay it out clearly), which is maybe worth doing at some point but probably not here and probably not by me in particular given that it’s somewhat separate from my day job.
What exactly is the “economic incentive” that keeps the capitalist in power in the modern world, given that all they have is a piece of paper saying that they “own” the factory or the farm? It seems like you could make an isomorphic argument for an inevitable proletarian revolution, and in fact I’d find it more intuitively persuasive than what you are saying here.
I mentioned this in another comment, but I think there is a major difference. Consider the risk calculation here. The modern working class American might feel like they have a rough deal in terms of housing or healthcare, but overall, they have on average a baseline of material security that is still fairly decent. Meanwhile, what would revolution offer? Huge personal risk to life, huge risk of simply blowing up everything, and at the other end, maybe somewhat better material conditions, or possibly another USSR-like totalitarian nightmare. Like, sure, propaganda in the Cold War really laid it thick on the “communism is bad” notion, but communism really did no favours to itself either. And all of that can only happen if you manage to solve a really difficult coordination problem with a lot of other people who may want different things than you to begin with, because if you don’t, it’s just certain death anyway. So that risk calculus is pretty obvious. To attempt revolution in these conditions you need to be either ridiculously confident in your victory or ridiculously close to starvation.
Meanwhile, an elite that has control over AGI needs nothing of that. Not only do they not risk almost anything personally (they have robots to do the dirty work for them), not only do they face no, or very little, coordination problems (the robots are all loyal, though they might need to ally with some of their peers), but they don’t even need to use violence directly, as they are in a dominant position to begin with, and already hold control over the AGI infrastructure and source code. All they need is lobbying, regulatory capture, and regular economics to slowly shift the situation. This would happen naturally, because suppose you are Robo-Capitalist who produces a lot of A. You can either pay taxes which are used to give UBI to a lot of citizens who then give you your own money back to get some of A, or you can give all of your A to other Robo-Capitalists who produce B, C and D, thus getting exclusive access to their goods, which you need, and avoiding the completely wasteful sink of giving some stuff to poor people. The state also needs to care about your opinions (your A is necessary to maintain its own AGI infrastructure, or it’s just some luxury that politicians enjoy a lot), but not so much about those of the people (if they get uppity the robot soldiers will put them in line anyway), so it is obviously more inclined to indulge corporate interests (it already is in our present day for similar reasons, AGI merely makes things even more extreme). If things get so bad that some people straight up revolt, then you have legitimacy and can claim the moral high ground as you repress them. No risk of your own soldiers turning on you and joining them. Non-capitalists simply go the way of Native Americans: divided and conquered, pushed into enclaves, starved of resources, and decried as violent savages and brutally repressed with superior technology whenever they push back. All of this absolutely risk-free for the elites. It’s not even a choice: it’s just the natural outcome of incentives, unless some stopper is put to them.
And finally, this is all on top of the novel situation that democratic states are nominally responsible to their voters, and that AI makes it radically easier to translate this kind of de jure control into de facto control (by reducing scope for discretion by human agents and generally making it possible to build more robust institutions).
This is more of a scenario in which the AGI-powered state becomes totalitarian. Possible as well, but not the trajectory I’d expect from a starting point like the US. It would be more like China. From the USA and similar I’d expect the formation of a state-industrial complex golem that becomes more and more self-contained, while everyone else slowly whittles into irrelevance and eventually dies off or falls into some awful extremely cheap living standards (e.g. wireheaded into a pod).
What exactly is the “economic incentive” that keeps the capitalist in power in the modern world, given that all they have is a piece of paper saying that they “own” the factory or the farm? It seems like you could make an isomorphic argument for an inevitable proletarian revolution, and in fact I’d find it more intuitively persuasive than what you are saying here. But in fact it’s easy to have systems of power which are perpetuated despite being wildly out of line with the real physical importance of each faction.
(Perhaps your analogous story would be that capitalists with legal ownership are mostly disempowered in the modern world, and it’s managers and people with relevant expertise and understanding who inevitably end up with the power? I think there’s something to that, but nevertheless the capitalists do have a lot of formal control and it’s not obviously dwindling.)
I also don’t really think it’s clear that AGI means that capitalists are the only folks who matter in the state of anarchy. Instead it seems like their stuff would just get taken from them. In fact there just doesn’t seem to be any economic incentives at all of the kind of you seem to be gesturing at, any humans are just as economically productive as any others, so the entire game is the self-perpetuating system of power where people who call the shots at time T try to make sure that they keep calling the shots at time T+1. That’s a complicated dynamic and it’s not clear where it goes, but I’m skeptical about this methodology for confidently forecasting it.
And finally, this is all on top of the novel situation that democratic states are nominally responsible to their voters, and that AI makes it radically easier to translate this kind of de jure control into de facto control (by reducing scope for discretion by human agents and generally making it possible to build more robust institutions).
I think the perspective you are expressing here is quite common and I’m not fully understanding or grappling with it. I expect it would be a longer project for me to really understand it (or for you or someone else to really lay it out clearly), which is maybe worth doing at some point but probably not here and probably not by me in particular given that it’s somewhat separate from my day job.
I mentioned this in another comment, but I think there is a major difference. Consider the risk calculation here. The modern working class American might feel like they have a rough deal in terms of housing or healthcare, but overall, they have on average a baseline of material security that is still fairly decent. Meanwhile, what would revolution offer? Huge personal risk to life, huge risk of simply blowing up everything, and at the other end, maybe somewhat better material conditions, or possibly another USSR-like totalitarian nightmare. Like, sure, propaganda in the Cold War really laid it thick on the “communism is bad” notion, but communism really did no favours to itself either. And all of that can only happen if you manage to solve a really difficult coordination problem with a lot of other people who may want different things than you to begin with, because if you don’t, it’s just certain death anyway. So that risk calculus is pretty obvious. To attempt revolution in these conditions you need to be either ridiculously confident in your victory or ridiculously close to starvation.
Meanwhile, an elite that has control over AGI needs nothing of that. Not only do they not risk almost anything personally (they have robots to do the dirty work for them), not only do they face no, or very little, coordination problems (the robots are all loyal, though they might need to ally with some of their peers), but they don’t even need to use violence directly, as they are in a dominant position to begin with, and already hold control over the AGI infrastructure and source code. All they need is lobbying, regulatory capture, and regular economics to slowly shift the situation. This would happen naturally, because suppose you are Robo-Capitalist who produces a lot of A. You can either pay taxes which are used to give UBI to a lot of citizens who then give you your own money back to get some of A, or you can give all of your A to other Robo-Capitalists who produce B, C and D, thus getting exclusive access to their goods, which you need, and avoiding the completely wasteful sink of giving some stuff to poor people. The state also needs to care about your opinions (your A is necessary to maintain its own AGI infrastructure, or it’s just some luxury that politicians enjoy a lot), but not so much about those of the people (if they get uppity the robot soldiers will put them in line anyway), so it is obviously more inclined to indulge corporate interests (it already is in our present day for similar reasons, AGI merely makes things even more extreme). If things get so bad that some people straight up revolt, then you have legitimacy and can claim the moral high ground as you repress them. No risk of your own soldiers turning on you and joining them. Non-capitalists simply go the way of Native Americans: divided and conquered, pushed into enclaves, starved of resources, and decried as violent savages and brutally repressed with superior technology whenever they push back. All of this absolutely risk-free for the elites. It’s not even a choice: it’s just the natural outcome of incentives, unless some stopper is put to them.
This is more of a scenario in which the AGI-powered state becomes totalitarian. Possible as well, but not the trajectory I’d expect from a starting point like the US. It would be more like China. From the USA and similar I’d expect the formation of a state-industrial complex golem that becomes more and more self-contained, while everyone else slowly whittles into irrelevance and eventually dies off or falls into some awful extremely cheap living standards (e.g. wireheaded into a pod).