That the only way and the right way to coexist with AGI is to embrace it maximally. That to achieve equity with this technology is to do the best we can to coordinate as many people as possible to have a personal relationship with it so that it can best reflect as diverse values and behaviors as possible. And in that we collaborate to define what it is to be a good human being and hope that our new technological child will heed us because it is us.
I also reject the notion that this level of coordination is impossible. This technology will truly be transformative but it appears that people who want access will have access. The productivity impact for the people who engage will outstrip the people who don’t. And the people most benefiting will determine the ‘culture’ of the AI. It’s only a matter of time before this awareness dawns on more and more of us because it will become the background context to the relationship. Expressing yourself to a future AI is like casting your personal vote towards an AGI that is somewhat like you.
I can conceive of that—and makes me wonder what parts we’ll have agency over and which we won’t. Do you or have you seen any appropriate labels between the difference that I could look up?