Thank you for expanding, I understand your position much better now :)
Prosocial behavior is very powerful, but if it were as powerful as you seem to expect, we wouldn’t need quite so many visionary leaders exhorting us not to be horrible to each other.
Where I think my optimistic viewpoint comes from in considering this related to superintelligence is that I think humans in general are prone to a bit of chaotic misunderstanding of their world. This makes the world… interesting… but to me also establishes a bit of a requirement for individuals who have a good understanding of the “bigger picture” to deploy some social control to stop everyone from going wild. As I type this I think about interesting parallels to the flood narrative/Noah’s Ark in the Book of Genesis.
With superintelligence, if architected correctly, we might be able to ensure that all/most of the most powerful intelligences in existence have a very accurate understanding of their world — without needing to encode and amplify specific values.
I agree they will have a very accurate understanding of the world, and will not have much difficulty arranging the world (humans included) according to their will. I’m not sure why that’s a source of optimism for you.
It may be because I believe that beauty, balance, and homeostasis are inherent in the world… if we have a powerful, intelligent system with deep understanding of this truth then I see a good future.
Thank you for expanding, I understand your position much better now :)
Where I think my optimistic viewpoint comes from in considering this related to superintelligence is that I think humans in general are prone to a bit of chaotic misunderstanding of their world. This makes the world… interesting… but to me also establishes a bit of a requirement for individuals who have a good understanding of the “bigger picture” to deploy some social control to stop everyone from going wild. As I type this I think about interesting parallels to the flood narrative/Noah’s Ark in the Book of Genesis.
With superintelligence, if architected correctly, we might be able to ensure that all/most of the most powerful intelligences in existence have a very accurate understanding of their world — without needing to encode and amplify specific values.
I agree they will have a very accurate understanding of the world, and will not have much difficulty arranging the world (humans included) according to their will. I’m not sure why that’s a source of optimism for you.
It may be because I believe that beauty, balance, and homeostasis are inherent in the world… if we have a powerful, intelligent system with deep understanding of this truth then I see a good future.