The economy can run just fine on instrumental desires of non-human entities (such as wanting to expand control) as the only source of wanting stuff. Given the choice between sinking the extra bit of resources into an arms race to expand control, and spending the same bit on feeding the comparatively unproductive human, the entity will choose the former every time.
The only way most people can keep getting fed despite being comparatively unproductive is by being a threat. But AI weaponry will greatly reduce the threat potential of big masses of people. The new balance of power will be more similar to what we had before firearms, when the powerful were free to treat most people really badly. And even worse, because this time around they won’t even need our labor.
Yes, such an outcome is possible. I think it’s unlikely, though, conditional on winding up in an Autofac-like world, because this requires a Terminator-style sudden takeover. If humans can see it coming or it happens gradually, they can react. And since we’re supposing here that there’s no AGI yet, there’s (1) very little reason for the humans directing economic activity to want this because it would also see their own elimination from the economy (even if they want to do something obvious like fully automate the money making machine that is their automated business, including the consumption side, they themselves still will want to consume, so there will remain demand for humans to consume within the economy, even if it becomes a small part of it), and (2) very little chance of AI coordinating a takeover like this on their own because, again, they don’t have their own motivations in this world.
The new balance of power will be more similar to what we had before firearms, when the powerful were free to treat most people really badly.
I expect the mediating force here to be the need for that mass of humans to be consumers driving the economy. Without them, growth would quickly stagnate or have to rely on bubbles, which aren’t sustainable.
I don’t think it requires a Terminator-style takeover. The obvious path is for AI to ally itself with money and power, leading to a world dominated by clumps of money+power+AI “goop”, maybe with a handful of people on top leading very nice lives. And it wouldn’t need the masses to provide market demand: anything it could get from the masses in exchange, it could instead produce on its own at lower resource cost.
And it wouldn’t need the masses to provide market demand: anything it could get from the masses in exchange, it could instead produce on its own at lower resource cost.
I think this still leaves us with the problem of, where does market demand to create growth come from? This small handful of people at the top will quickly reach max comfort lives and then not need anything else, but humans usually want more, even if that more is just a number going up, we’re back to either quick stagnation or a bubble.
One good that might be offered is dominance. I didn’t think of this before, but we could imagine a world where the “underclass” receive UBI in exchange for fealty, and the folks at the top compete to see who can have the most vassals, with intense competition to attract and keep vassals driving economic growth.
“As material needs are increasingly met, more surplus goes into positional goods” seems like a common pattern indeed. Note “positional” and not purely “luxury”. I consider both prestige-status and dominance-status to be associated with position here. Even the former, while it could lead to a competition for loyalty that’s more like a gift economy and cooperates with the underclass, could also lead to elites trying to outcompete each other in pure consumption such that “max comfort” stops being a limit for how much they want compared to the underclass. Indeed I vaguely recall hearing that such a dynamic, where status is associated with how much you can visibly spend, already holds among some elite classes in the current day.
My thoughts are shaped by the cultural waves of the last few decades in the USA, so they lean toward imagining the moral fig leaf as something like “make sure the people benefiting from our stuff aren’t enemies/saboteurs/Problematic” and a gradual expansion of what counts as the excluded people that involves an increasing amount of political and psychological boxing-in of the remainder. That all flows nicely with the “find ways to get potential rebels to turn each other in” sort of approach too. Of course that’s one of many ways that a dominance-motivated persistent class asymmetry could play out.
If you’re familiar with the story “The Metamorphosis of Prime Intellect”, a shorter story that the author wrote in the same universe, “A Casino Odyssey in Cyberspace”, depicts characters with some related tendencies playing out against a backdrop of wild surplus in a way you may find stimulating to the imagination. (Edited to add: both stories are also heavy on sex and violence and such, so be cautious if you find such things disturbing.) Also, Kurt Vonnegut’s novel Player Piano about a world without need of much human labor doesn’t show the harsh version I have in mind, but the way ‘sabotage’ is treated evokes a subtler form of repression (maybe not worth reading all of just for this though).
I already answered it in the first comment though. These big clumps of money+power+AI will have convergent instrumental goals in Omohundro’s sense. They’ll want expansion, control, arms races. That’s quite enough motivation for growth.
About the idea of the underclass receiving UBI and having a right to choose their masters—I think this was also covered in the first comment. There will be no UBI and no rights, because the underclass will have no threat potential. Most likely the underclass will be just discarded. Or if some masters want dominance, they’ll get it by force; it’s been like that for most of history.
I think it’s likely that people will be enough of a threat to prevent the kind of outcome you’re proposing, which is why I think this model is interesting. You seem to disagree. Would you agree that that’s the crux of why we disagree?
(I don’t disagree that what you say is possible. I just think it’s not very likely to happen and hence not the “happy” path of my model.)
I don’t know how much time you spend thinking about the distribution of power, roughly speaking between the masses and the money+power clumps. For me in the past few years it’s been a lot. You could call it becoming “woke” in the original sense of the word; this awareness seems to be more a thing on the left. And the more I look, the more I see the balance is tilting away from the masses. AI weapons would be the final nail, but maybe they aren’t even necessary; maybe divide-and-conquer manipulation slightly stronger than today will be already enough to neutralize the threat of the masses completely.
The economy can run just fine on instrumental desires of non-human entities (such as wanting to expand control) as the only source of wanting stuff. Given the choice between sinking the extra bit of resources into an arms race to expand control, and spending the same bit on feeding the comparatively unproductive human, the entity will choose the former every time.
The only way most people can keep getting fed despite being comparatively unproductive is by being a threat. But AI weaponry will greatly reduce the threat potential of big masses of people. The new balance of power will be more similar to what we had before firearms, when the powerful were free to treat most people really badly. And even worse, because this time around they won’t even need our labor.
Yes, such an outcome is possible. I think it’s unlikely, though, conditional on winding up in an Autofac-like world, because this requires a Terminator-style sudden takeover. If humans can see it coming or it happens gradually, they can react. And since we’re supposing here that there’s no AGI yet, there’s (1) very little reason for the humans directing economic activity to want this because it would also see their own elimination from the economy (even if they want to do something obvious like fully automate the money making machine that is their automated business, including the consumption side, they themselves still will want to consume, so there will remain demand for humans to consume within the economy, even if it becomes a small part of it), and (2) very little chance of AI coordinating a takeover like this on their own because, again, they don’t have their own motivations in this world.
I expect the mediating force here to be the need for that mass of humans to be consumers driving the economy. Without them, growth would quickly stagnate or have to rely on bubbles, which aren’t sustainable.
I don’t think it requires a Terminator-style takeover. The obvious path is for AI to ally itself with money and power, leading to a world dominated by clumps of money+power+AI “goop”, maybe with a handful of people on top leading very nice lives. And it wouldn’t need the masses to provide market demand: anything it could get from the masses in exchange, it could instead produce on its own at lower resource cost.
I think this still leaves us with the problem of, where does market demand to create growth come from? This small handful of people at the top will quickly reach max comfort lives and then not need anything else, but humans usually want more, even if that more is just a number going up, we’re back to either quick stagnation or a bubble.
One good that might be offered is dominance. I didn’t think of this before, but we could imagine a world where the “underclass” receive UBI in exchange for fealty, and the folks at the top compete to see who can have the most vassals, with intense competition to attract and keep vassals driving economic growth.
“As material needs are increasingly met, more surplus goes into positional goods” seems like a common pattern indeed. Note “positional” and not purely “luxury”. I consider both prestige-status and dominance-status to be associated with position here. Even the former, while it could lead to a competition for loyalty that’s more like a gift economy and cooperates with the underclass, could also lead to elites trying to outcompete each other in pure consumption such that “max comfort” stops being a limit for how much they want compared to the underclass. Indeed I vaguely recall hearing that such a dynamic, where status is associated with how much you can visibly spend, already holds among some elite classes in the current day.
My thoughts are shaped by the cultural waves of the last few decades in the USA, so they lean toward imagining the moral fig leaf as something like “make sure the people benefiting from our stuff aren’t enemies/saboteurs/Problematic” and a gradual expansion of what counts as the excluded people that involves an increasing amount of political and psychological boxing-in of the remainder. That all flows nicely with the “find ways to get potential rebels to turn each other in” sort of approach too. Of course that’s one of many ways that a dominance-motivated persistent class asymmetry could play out.
If you’re familiar with the story “The Metamorphosis of Prime Intellect”, a shorter story that the author wrote in the same universe, “A Casino Odyssey in Cyberspace”, depicts characters with some related tendencies playing out against a backdrop of wild surplus in a way you may find stimulating to the imagination. (Edited to add: both stories are also heavy on sex and violence and such, so be cautious if you find such things disturbing.) Also, Kurt Vonnegut’s novel Player Piano about a world without need of much human labor doesn’t show the harsh version I have in mind, but the way ‘sabotage’ is treated evokes a subtler form of repression (maybe not worth reading all of just for this though).
I already answered it in the first comment though. These big clumps of money+power+AI will have convergent instrumental goals in Omohundro’s sense. They’ll want expansion, control, arms races. That’s quite enough motivation for growth.
About the idea of the underclass receiving UBI and having a right to choose their masters—I think this was also covered in the first comment. There will be no UBI and no rights, because the underclass will have no threat potential. Most likely the underclass will be just discarded. Or if some masters want dominance, they’ll get it by force; it’s been like that for most of history.
I think it’s likely that people will be enough of a threat to prevent the kind of outcome you’re proposing, which is why I think this model is interesting. You seem to disagree. Would you agree that that’s the crux of why we disagree?
(I don’t disagree that what you say is possible. I just think it’s not very likely to happen and hence not the “happy” path of my model.)
It’s the crux, yeah.
I don’t know how much time you spend thinking about the distribution of power, roughly speaking between the masses and the money+power clumps. For me in the past few years it’s been a lot. You could call it becoming “woke” in the original sense of the word; this awareness seems to be more a thing on the left. And the more I look, the more I see the balance is tilting away from the masses. AI weapons would be the final nail, but maybe they aren’t even necessary; maybe divide-and-conquer manipulation slightly stronger than today will be already enough to neutralize the threat of the masses completely.
Why stop with fealty? Billionaire-barons could demand votes for income.