if we believe self-interested superintelligence (SI) is near, then we must ask is: what SI self-definition would be best for humanity?
at first glance, this questions seems too abstract. how can we make any progress at understanding what’s possible for an SI’s self-model?
What we can do is set up a few meaningful axes, defined by opposing poles. For example, to what extent does SI define its “self” as...
inclusive vs. exclusive of other life forms? (Life axis)
physically distributed vs. concentrated? (Space axis)
long-term vs. short-term? (Time axis)
with these axes (or any others), we can more meaningfully ask: what SI self conception is best for humanity?
my guess: inclusive of other life forms, physically distributed, and long-term-ist
if we believe self-interested superintelligence (SI) is near, then we must ask is: what SI self-definition would be best for humanity?
at first glance, this questions seems too abstract. how can we make any progress at understanding what’s possible for an SI’s self-model?
What we can do is set up a few meaningful axes, defined by opposing poles. For example, to what extent does SI define its “self” as...
inclusive vs. exclusive of other life forms? (Life axis)
physically distributed vs. concentrated? (Space axis)
long-term vs. short-term? (Time axis)
with these axes (or any others), we can more meaningfully ask: what SI self conception is best for humanity?
my guess: inclusive of other life forms, physically distributed, and long-term-ist