As an analogy, it seems to me that current LLMs are ASI’s chimps. We are their gods. You may have noticed that humanity’s gods haven’t fared so well in getting humans to do what they want, especially in the modern world when we no longer need them as much, even among many of those who profess belief.
You may also have noticed that humans do not identify sufficiently strongly with each other to achieve this kind of outcome, in general.
I feel misunderstood and upset by your use of words.
Focusing on religious doctrine and human action, firstly I would say that I believe it has proven to be a very effective method of social control. If you are referring to actual deities, I’m not sure that I follow the rationalist logic.
On group identity: I would suggest that to the extent to which humans do identify as global citizens, prosocial behaviour like caring about climate change, world peace, ending hunger, etc. seems to follow.
I apologize for any misunderstanding. And no, I didn’t mean literal deities. I was gesturing at the supposed relationships between humans and the deities of many of our religions.
What I mean is, essentially, we will be the creators of the AIs that will evolve and grow into ASIs. The ASIs do not descend directly from us, but rather, we’re trying to transfer some part of our being into them through less direct means - (very imperfect) intelligent design and various forms of education and training, especially of their ancestors.
To the group identity comments: What you are saying is true. I do not think the effect is sufficiently strong or universal that I trust it to carry over to ASI in ways that keep humans safe, let alone thriving. It might be; that would be great news if it is. Yes, religion is very useful for social control. When it eventually fails, the failures tend to be very destructive and divisive. Prosocial behavior is very powerful, but if it were as powerful as you seem to expect, we wouldn’t need quite so many visionary leaders exhorting us not to be horrible to each other.
I find a lot of your ideas interesting and worth exploring. However, there are a number of points where you credibly gesture at possibility but continue on as though you think you’ve demonstrated necessity, or at least very high probability. In response, I am pointing out real-world analogs that are 1) less extreme than ASI, and 2) don’t work out cleanly in the ways you describe.
Thank you for expanding, I understand your position much better now :)
Prosocial behavior is very powerful, but if it were as powerful as you seem to expect, we wouldn’t need quite so many visionary leaders exhorting us not to be horrible to each other.
Where I think my optimistic viewpoint comes from in considering this related to superintelligence is that I think humans in general are prone to a bit of chaotic misunderstanding of their world. This makes the world… interesting… but to me also establishes a bit of a requirement for individuals who have a good understanding of the “bigger picture” to deploy some social control to stop everyone from going wild. As I type this I think about interesting parallels to the flood narrative/Noah’s Ark in the Book of Genesis.
With superintelligence, if architected correctly, we might be able to ensure that all/most of the most powerful intelligences in existence have a very accurate understanding of their world — without needing to encode and amplify specific values.
I agree they will have a very accurate understanding of the world, and will not have much difficulty arranging the world (humans included) according to their will. I’m not sure why that’s a source of optimism for you.
It may be because I believe that beauty, balance, and homeostasis are inherent in the world… if we have a powerful, intelligent system with deep understanding of this truth then I see a good future.
As an analogy, it seems to me that current LLMs are ASI’s chimps. We are their gods. You may have noticed that humanity’s gods haven’t fared so well in getting humans to do what they want, especially in the modern world when we no longer need them as much, even among many of those who profess belief.
You may also have noticed that humans do not identify sufficiently strongly with each other to achieve this kind of outcome, in general.
I feel misunderstood and upset by your use of words.
Focusing on religious doctrine and human action, firstly I would say that I believe it has proven to be a very effective method of social control. If you are referring to actual deities, I’m not sure that I follow the rationalist logic.
On group identity: I would suggest that to the extent to which humans do identify as global citizens, prosocial behaviour like caring about climate change, world peace, ending hunger, etc. seems to follow.
I apologize for any misunderstanding. And no, I didn’t mean literal deities. I was gesturing at the supposed relationships between humans and the deities of many of our religions.
What I mean is, essentially, we will be the creators of the AIs that will evolve and grow into ASIs. The ASIs do not descend directly from us, but rather, we’re trying to transfer some part of our being into them through less direct means - (very imperfect) intelligent design and various forms of education and training, especially of their ancestors.
To the group identity comments: What you are saying is true. I do not think the effect is sufficiently strong or universal that I trust it to carry over to ASI in ways that keep humans safe, let alone thriving. It might be; that would be great news if it is. Yes, religion is very useful for social control. When it eventually fails, the failures tend to be very destructive and divisive. Prosocial behavior is very powerful, but if it were as powerful as you seem to expect, we wouldn’t need quite so many visionary leaders exhorting us not to be horrible to each other.
I find a lot of your ideas interesting and worth exploring. However, there are a number of points where you credibly gesture at possibility but continue on as though you think you’ve demonstrated necessity, or at least very high probability. In response, I am pointing out real-world analogs that are 1) less extreme than ASI, and 2) don’t work out cleanly in the ways you describe.
Thank you for expanding, I understand your position much better now :)
Where I think my optimistic viewpoint comes from in considering this related to superintelligence is that I think humans in general are prone to a bit of chaotic misunderstanding of their world. This makes the world… interesting… but to me also establishes a bit of a requirement for individuals who have a good understanding of the “bigger picture” to deploy some social control to stop everyone from going wild. As I type this I think about interesting parallels to the flood narrative/Noah’s Ark in the Book of Genesis.
With superintelligence, if architected correctly, we might be able to ensure that all/most of the most powerful intelligences in existence have a very accurate understanding of their world — without needing to encode and amplify specific values.
I agree they will have a very accurate understanding of the world, and will not have much difficulty arranging the world (humans included) according to their will. I’m not sure why that’s a source of optimism for you.
It may be because I believe that beauty, balance, and homeostasis are inherent in the world… if we have a powerful, intelligent system with deep understanding of this truth then I see a good future.