I think you do a good job at expanding the possible set of self conceptions that we could reasonably expect in AIs.
Your discussion of these possible selves inspires me to go farther than you in your recommendations for AI safety researchers. Stress testing safety ideas across multiple different possible “selfs” is good. But, if an AI’s individuality/self determines to a great degree its behavior and growth, then safety research as a whole might be better conceived as an effort to influence AI’s self conceptions rather than control their resulting behavior. E.g., create seed conditions that make it more likely for AIs to identify with people, to include people within its “individuality,” than to identify only with other machines.
I think you do a good job at expanding the possible set of self conceptions that we could reasonably expect in AIs.
Your discussion of these possible selves inspires me to go farther than you in your recommendations for AI safety researchers. Stress testing safety ideas across multiple different possible “selfs” is good. But, if an AI’s individuality/self determines to a great degree its behavior and growth, then safety research as a whole might be better conceived as an effort to influence AI’s self conceptions rather than control their resulting behavior. E.g., create seed conditions that make it more likely for AIs to identify with people, to include people within its “individuality,” than to identify only with other machines.