I see. I think about 99% of humanity at the very least are not so sadistic as to create a future with less than zero utility. Sociopaths are something like ten percent of the population, but like everything else it’s on a spectrum. Sociopaths usually also have some measure of empathy and desire for approval. In a world where they’ve won, I think most of them would rather be bailed as a hero than be an eternal sadistic despot. Sociopathathy doesn’t automatically include a lot of sadism, just desire for revenge against perceived enemies.
So I’d take my chances with a human overlord far before accepting extinction.
Note that our light cone with zero value might also eclipse other light cones that might’ve had value if we didn’t let our AGI go rogue to avoid s-risk.
Note that our light cone with zero value might also eclipse other light cones that might’ve had value if we didn’t let our AGI go rogue to avoid s-risk.
That’s a good thing to consider! However, taking Earth’s situation as a prior for other “cradles of intelligence”, I think that consideration returns to the question of “should we expect Earth’s lightcone to be better or worse than zero-value (conditional on corrigibility)?”
IIUC, your model would (at least tentatively) predict that
if person P has a lot of power over person Q,
and P is not sadistic,
and P is sufficiently secure/well-resourced that P doesn’t “need” to exploit Q,
then P will not intentionally do anything that would be horrible for Q?
If so, how do you reconcile that with e.g. non-sadistic serial killers, rapists, or child abusers? Or non-sadistic narcissists in whose ideal world everyone else would be their worshipful subject/slave?
That last point also raises the question: Would you prefer the existence of lots of (either happily or grudgingly) submissive slaves over oblivion?
To me it seems that terrible outcomes do not require sadism. Seems sufficient that P be low in empathy, and want from Q something Q does not want to provide (like admiration, submission, sex, violent sport, or even just attention).[1] I’m confused as to how/why you disagree.
Also, AFAICT, about 0.5% to 8% of humans are sadistic, and about 8% to 16% have very little or zero empathy. How did you arrive at “99% of humanity [...] are not so sadistic”? Did you account for the fact that most people with sadistic inclinations probably try to hide those inclinations? (Like, if only 0.5% of people appear sadistic, then I’d expect the actual prevalence of sadism to be more like ~4%.)
I see. I think about 99% of humanity at the very least are not so sadistic as to create a future with less than zero utility. Sociopaths are something like ten percent of the population, but like everything else it’s on a spectrum. Sociopaths usually also have some measure of empathy and desire for approval. In a world where they’ve won, I think most of them would rather be bailed as a hero than be an eternal sadistic despot. Sociopathathy doesn’t automatically include a lot of sadism, just desire for revenge against perceived enemies.
So I’d take my chances with a human overlord far before accepting extinction.
Note that our light cone with zero value might also eclipse other light cones that might’ve had value if we didn’t let our AGI go rogue to avoid s-risk.
That’s a good thing to consider! However, taking Earth’s situation as a prior for other “cradles of intelligence”, I think that consideration returns to the question of “should we expect Earth’s lightcone to be better or worse than zero-value (conditional on corrigibility)?”
IIUC, your model would (at least tentatively) predict that
if person P has a lot of power over person Q,
and P is not sadistic,
and P is sufficiently secure/well-resourced that P doesn’t “need” to exploit Q,
then P will not intentionally do anything that would be horrible for Q?
If so, how do you reconcile that with e.g. non-sadistic serial killers, rapists, or child abusers? Or non-sadistic narcissists in whose ideal world everyone else would be their worshipful subject/slave?
That last point also raises the question: Would you prefer the existence of lots of (either happily or grudgingly) submissive slaves over oblivion?
To me it seems that terrible outcomes do not require sadism. Seems sufficient that P be low in empathy, and want from Q something Q does not want to provide (like admiration, submission, sex, violent sport, or even just attention).[1] I’m confused as to how/why you disagree.
Also, AFAICT, about 0.5% to 8% of humans are sadistic, and about 8% to 16% have very little or zero empathy. How did you arrive at “99% of humanity [...] are not so sadistic”? Did you account for the fact that most people with sadistic inclinations probably try to hide those inclinations? (Like, if only 0.5% of people appear sadistic, then I’d expect the actual prevalence of sadism to be more like ~4%.)