It seems incredibly unlikely that human social instincts can only be satisfied – or even can be best satisfied – by other humans.
You’re 100% not understanding my argument, which is sorta fair because I didn’t lay it out clearly, but I think you should be doing better anyway.
Here’s a sketch:
Humans want to be human-ish and be around human-ish entities.
So the emperor will be human-ish and be around human-ish entities for a long time. (Ok, to be clear, I mean a lot of developmental / experiential time—the thing that’s relevant for thinking about how the emperor’s way of being trends over time.)
When being human-ish and around human-ish entities, core human shards continue to work.
When core human shards continue to work, MAYBE this implies EVENTUALLY adopting beneficence (or something else like cosmopolitanism), and hence good outcomes.
Since the emperor will be human-ish and be around human-ish entities for a long time, IF 4 obtains, then good outomes.
And then I give two IDEAS about 4 (communing->[universalist democracy], and [information increases]->understanding->caring).
I don’t know what’s making you think I don’t understand your argument. Also, I’ve never publicly stated that I’m opting into Crocker’s Rules, so while I happen not to particularly mind the rudeness, your general policy on that seems out of line here.
When being human-ish and around human-ish entities, core human shards continue to work
My argument is that the process you’re hypothesizing would be sensitive to the exact way of being human-ish, the exact classes of human-ish entities around, and the exact circumstances in which the emperor has to be around them.
As a plain and down-to-earth example, if a racist surrounds themselves with a hand-picked group of racist friends, do you expect them to eventually develop universal empathy, solely through interacting with said racist friends? Addressing your specific ideas: nobody in that group would ever need to commune with non-racists, nor have to bother learning more about non-racists. And empirically, such groups don’t seem to undergo spontaneous deradicalizations.
As a plain and down-to-earth example, if a racist surrounds themselves with a hand-picked group of racist friends, do you expect them to eventually develop universal empathy, solely through interacting with said racist friends? Addressing your specific ideas: nobody in that group would ever need to commune with non-racists, nor have to bother learning more about non-racists. And empirically, such groups don’t seem to undergo spontaneous deradicalizations.
As a plain and down-to-earth example, if a racist surrounds themselves with a hand-picked group of racist friends, do you expect them to eventually develop universal empathy, solely through interacting with said racist friends? Addressing your specific ideas: nobody in that group would ever need to commune with non-racists, nor have to bother learning more about non-racists. And empirically, such groups don’t seem to undergo spontaneous deradicalizations.
So what do you think happens when they are hanging out together, and they are in charge, and it has been 1,000 years or 1,000,000 years?
They keep each other radicalized forever as part of some transcendental social dynamic.
They become increasingly non-human as time goes on, small incremental modifications and personality changes building on each other, until they’re no longer human in the senses necessary for your hypothesis to apply.
I assume your counter-model involves them getting bored of each other and seeking diversity/new friends, or generating new worlds to explore/communicate with, with the generating processes not constrained to only generate racists, leading to the extremists interacting with non-extremists and eventually incrementally adopting non-extremist perspectives?
If yes, this doesn’t seem like the overdetermined way for things to go:
The generating processes would likely be skewed towards only generating things the extremists would find palatable, meaning more people sharing their perspectives/not seriously challenging whatever deeply seated prejudices they have. They’re there to have a good time, not have existential/moral crises.
They may make any number of modifications to themselves to make them no longer human-y in the relevant sense. Including by simply letting human-standard self-modification algorithms run for 10^3-10^6 years, becoming superhumanly radicalized.
They may address the “getting bored” part instead, periodically wiping their memories (including by standard human forgetting) or increasing each other’s capacity to generate diverse interactions.
Ok so they only generate racists and racially pure people. And they do their thing. But like, there’s no other races around, so the racism part sorta falls by the wayside. They’re still racially pure of course, but it’s usually hard to tell that they’re racist; sometimes they sit around and make jokes to feel superior over lesser races, but this is pretty hollow since they’re not really engaged in any type of race relations. Their world isn’t especially about all that, anymore. Now it’s about… what? I don’t know what to imagine here, but the only things I do know how to imagine involve unbounded structure (e.g. math, art, self-reflection, self-reprogramming). So, they’re doing that stuff. For a very long time. And the race thing just is not a part of their world anymore. Or is it? I don’t even know what to imagine there. Instead of having tastes about ethnicity, they develop tastes about questions in math, or literature. In other words, [the differences between people and groups that they care about] migrate from race to features of people that are involved in unbounded stuff. If the AGI has been keeping the racially impure in an enclosure all this time, at some point the racists might have a glance back, and say, wait, all the interesting stuff about people is also interesting about these people. Why not have them join us as well.
You’re 100% not understanding my argument, which is sorta fair because I didn’t lay it out clearly, but I think you should be doing better anyway.
Here’s a sketch:
Humans want to be human-ish and be around human-ish entities.
So the emperor will be human-ish and be around human-ish entities for a long time. (Ok, to be clear, I mean a lot of developmental / experiential time—the thing that’s relevant for thinking about how the emperor’s way of being trends over time.)
When being human-ish and around human-ish entities, core human shards continue to work.
When core human shards continue to work, MAYBE this implies EVENTUALLY adopting beneficence (or something else like cosmopolitanism), and hence good outcomes.
Since the emperor will be human-ish and be around human-ish entities for a long time, IF 4 obtains, then good outomes.
And then I give two IDEAS about 4 (communing->[universalist democracy], and [information increases]->understanding->caring).
I don’t know what’s making you think I don’t understand your argument. Also, I’ve never publicly stated that I’m opting into Crocker’s Rules, so while I happen not to particularly mind the rudeness, your general policy on that seems out of line here.
My argument is that the process you’re hypothesizing would be sensitive to the exact way of being human-ish, the exact classes of human-ish entities around, and the exact circumstances in which the emperor has to be around them.
As a plain and down-to-earth example, if a racist surrounds themselves with a hand-picked group of racist friends, do you expect them to eventually develop universal empathy, solely through interacting with said racist friends? Addressing your specific ideas: nobody in that group would ever need to commune with non-racists, nor have to bother learning more about non-racists. And empirically, such groups don’t seem to undergo spontaneous deradicalizations.
I expect they’d get bored with that.
So what do you think happens when they are hanging out together, and they are in charge, and it has been 1,000 years or 1,000,000 years?
One or both of:
They keep each other radicalized forever as part of some transcendental social dynamic.
They become increasingly non-human as time goes on, small incremental modifications and personality changes building on each other, until they’re no longer human in the senses necessary for your hypothesis to apply.
I assume your counter-model involves them getting bored of each other and seeking diversity/new friends, or generating new worlds to explore/communicate with, with the generating processes not constrained to only generate racists, leading to the extremists interacting with non-extremists and eventually incrementally adopting non-extremist perspectives?
If yes, this doesn’t seem like the overdetermined way for things to go:
The generating processes would likely be skewed towards only generating things the extremists would find palatable, meaning more people sharing their perspectives/not seriously challenging whatever deeply seated prejudices they have. They’re there to have a good time, not have existential/moral crises.
They may make any number of modifications to themselves to make them no longer human-y in the relevant sense. Including by simply letting human-standard self-modification algorithms run for 10^3-10^6 years, becoming superhumanly radicalized.
They may address the “getting bored” part instead, periodically wiping their memories (including by standard human forgetting) or increasing each other’s capacity to generate diverse interactions.
Ok so they only generate racists and racially pure people. And they do their thing. But like, there’s no other races around, so the racism part sorta falls by the wayside. They’re still racially pure of course, but it’s usually hard to tell that they’re racist; sometimes they sit around and make jokes to feel superior over lesser races, but this is pretty hollow since they’re not really engaged in any type of race relations. Their world isn’t especially about all that, anymore. Now it’s about… what? I don’t know what to imagine here, but the only things I do know how to imagine involve unbounded structure (e.g. math, art, self-reflection, self-reprogramming). So, they’re doing that stuff. For a very long time. And the race thing just is not a part of their world anymore. Or is it? I don’t even know what to imagine there. Instead of having tastes about ethnicity, they develop tastes about questions in math, or literature. In other words, [the differences between people and groups that they care about] migrate from race to features of people that are involved in unbounded stuff. If the AGI has been keeping the racially impure in an enclosure all this time, at some point the racists might have a glance back, and say, wait, all the interesting stuff about people is also interesting about these people. Why not have them join us as well.