The economic rationale for human servitude disappears when the machines are better than humans at everything. That doesn’t prevent sadistic mistreatment or killing the poor to use their atoms for something else, but it’s a major disanalogy from history. What lessons you draw probably depend on whether you think the rich and powerful are sadistic (actively wanting to harm the poor) or merely mostly selfish (happy to help the poor if it’s trivially cheap or they get their name on a building in return, but not if it makes a dent in the yacht and caviar budget).
“Actively wanting to harm the poor” doesn’t strike at the heart of the issue. Nor is it about economics. The issue is that the powerful want to feel socially dominant. There have been plenty of historical examples where this turned ugly.
I’m maybe more attuned to this than most people. I still remember my first time (as a child) going to a restaurant that had waiters, and feeling very clearly that being waited-on was not only about getting food, but also partly an ugly dominance ritual that I wanted no part of. On the same continuum you have kings forcing subjects to address them as “Your Majesty”: it still kinda blows my mind that that was a real thing.
I see. I think you should write a post trying to imagine in detail the failure modes you foresee if AI is aligned to the rich and powerful. What happens to the masses in those worlds, specifically? Are they killed, tortured, forced to work as waiters, or what? I have “merely mostly selfish” psych intuitions, so when I imagine Sam Altman being God-Emperor, I imagine that being like “luxury post-scarcity utopia except everyone has been brainwashed to express gratitude to the God-Emperor Sam I for giving them eternal life in utopia”, which is not ideal, but still arguably vastly better than worlds (like the status quo) with death and suffering. If you’re envisioning something darker, I think being more concrete would help puncture the optimism of people like me.
Hmm hm. Being forced to play out a war? Getting people’s minds modified so they behave like house elves from HP? Selective breeding? Selling some of your poor people to another rich person who’d like to have them? It’s not even like I’m envisioning something specific that’s dark, I just know that a world where some human beings have absolute root-level power over many others is gonna be dark. Let’s please not build such a world.
The powerful want to be socially dominant, but to what extent are they willing to engineer painful conditions to experience a greater high from the quality of life disparity? In a world with robots delivering extreme material abundance, this kind of is “actively wanting to harm the poor”. It’s true that some sadistically enjoy signs of disparity, but how much of that is extracting pleasure from the economic realities compared to it being the intrinsic motivation for the power?
I’m not sure on what the right way to model how this will play out, but my guess is that the outcome isn’t knowable from where we stand. I think it will heavily depend on:
The particular predispositions of the powerful people pushing the technology
The economic rationale for human servitude disappears when the machines are better than humans at everything. That doesn’t prevent sadistic mistreatment or killing the poor to use their atoms for something else, but it’s a major disanalogy from history. What lessons you draw probably depend on whether you think the rich and powerful are sadistic (actively wanting to harm the poor) or merely mostly selfish (happy to help the poor if it’s trivially cheap or they get their name on a building in return, but not if it makes a dent in the yacht and caviar budget).
“Actively wanting to harm the poor” doesn’t strike at the heart of the issue. Nor is it about economics. The issue is that the powerful want to feel socially dominant. There have been plenty of historical examples where this turned ugly.
I’m maybe more attuned to this than most people. I still remember my first time (as a child) going to a restaurant that had waiters, and feeling very clearly that being waited-on was not only about getting food, but also partly an ugly dominance ritual that I wanted no part of. On the same continuum you have kings forcing subjects to address them as “Your Majesty”: it still kinda blows my mind that that was a real thing.
I see. I think you should write a post trying to imagine in detail the failure modes you foresee if AI is aligned to the rich and powerful. What happens to the masses in those worlds, specifically? Are they killed, tortured, forced to work as waiters, or what? I have “merely mostly selfish” psych intuitions, so when I imagine Sam Altman being God-Emperor, I imagine that being like “luxury post-scarcity utopia except everyone has been brainwashed to express gratitude to the God-Emperor Sam I for giving them eternal life in utopia”, which is not ideal, but still arguably vastly better than worlds (like the status quo) with death and suffering. If you’re envisioning something darker, I think being more concrete would help puncture the optimism of people like me.
Hmm hm. Being forced to play out a war? Getting people’s minds modified so they behave like house elves from HP? Selective breeding? Selling some of your poor people to another rich person who’d like to have them? It’s not even like I’m envisioning something specific that’s dark, I just know that a world where some human beings have absolute root-level power over many others is gonna be dark. Let’s please not build such a world.
The powerful want to be socially dominant, but to what extent are they willing to engineer painful conditions to experience a greater high from the quality of life disparity? In a world with robots delivering extreme material abundance, this kind of is “actively wanting to harm the poor”. It’s true that some sadistically enjoy signs of disparity, but how much of that is extracting pleasure from the economic realities compared to it being the intrinsic motivation for the power?
I’m not sure on what the right way to model how this will play out, but my guess is that the outcome isn’t knowable from where we stand. I think it will heavily depend on:
The particular predispositions of the powerful people pushing the technology
The shape of the tech tree and how we explore it