an existential battle between human executive function and ourselves… eventually humanity loses its mind as the boundaries of reality become irreconcilable
This description is confusing, but I assume you’re talking about a process in which decision-making in a human-AI hybrid ends up entirely in the AI part rather than the human part.
It’s logical to worry about such a thing because AI is faster than human already. However, if we actually knew what we were doing, perhaps AI superintelligence could be incorporated into an augmented human, in such a way that there is continuity of control. Wherever the executive function or the Cartesian theater is localized, maybe you can migrate it onto a faster substrate, or give it accelerated “reflexes” which mediate between human-speed conscious decision-making and faster-than-human superintelligent subsystems… But we don’t know enough to do more than speculate at this point.
For the big picture, your items 1 and 2 could be joined by choice 3 (don’t make AI) and non-choice 4 (the AI takes over and makes the decisions). I think we’re headed for 4, personally, in which case you want to solve alignment in the sense that applies to an autonomous superintelligence.
Thanks for the comment. I was imprecise with the “boundaries of reality” framing but beyond individual physical boundaries (human-AI hybrids) I’m also talking about boundaries within the fabric of social and cultural life: entertainment media, political narratives, world models. As this is influenced more by AI, I think we lose human identity.
4) to me falls under 2) as it encompasses AI free from human supervision and able permeate all aspects of life.
This description is confusing, but I assume you’re talking about a process in which decision-making in a human-AI hybrid ends up entirely in the AI part rather than the human part.
It’s logical to worry about such a thing because AI is faster than human already. However, if we actually knew what we were doing, perhaps AI superintelligence could be incorporated into an augmented human, in such a way that there is continuity of control. Wherever the executive function or the Cartesian theater is localized, maybe you can migrate it onto a faster substrate, or give it accelerated “reflexes” which mediate between human-speed conscious decision-making and faster-than-human superintelligent subsystems… But we don’t know enough to do more than speculate at this point.
For the big picture, your items 1 and 2 could be joined by choice 3 (don’t make AI) and non-choice 4 (the AI takes over and makes the decisions). I think we’re headed for 4, personally, in which case you want to solve alignment in the sense that applies to an autonomous superintelligence.
Thanks for the comment. I was imprecise with the “boundaries of reality” framing but beyond individual physical boundaries (human-AI hybrids) I’m also talking about boundaries within the fabric of social and cultural life: entertainment media, political narratives, world models. As this is influenced more by AI, I think we lose human identity.
4) to me falls under 2) as it encompasses AI free from human supervision and able permeate all aspects of life.