“Scale-free, physical” ethics (notably, interacting with naturalistic theories of consciousness rather than phenomenological, hermeneutic, and other “continental” studies and accounts of consciousness)
Clearly, technical AI alignment cannot take a “human” with some formulaic definition as only or primary moral subjects and/or subjects to be aligned with. This itself would be unscientific. Rather, alignment should be based on some naturalistic theory of ethics, e.g., saying that moral subjectivity is proportional to integrated informationΦ in the agent’s consciousness. The “values” are also determined scientifically, from the game-theoretic/evolutionary setup.
So, a cyborg will also be a subject of alignment. But it also extends moral/alignment subjectivity to animals, of course.
I think the term alignment subjectivity wasn’t used before, looks like a useful term, let’s coin it :)
It’s somewhere between the lines here:
Clearly, technical AI alignment cannot take a “human” with some formulaic definition as only or primary moral subjects and/or subjects to be aligned with. This itself would be unscientific. Rather, alignment should be based on some naturalistic theory of ethics, e.g., saying that moral subjectivity is proportional to integrated information Φ in the agent’s consciousness. The “values” are also determined scientifically, from the game-theoretic/evolutionary setup.
So, a cyborg will also be a subject of alignment. But it also extends moral/alignment subjectivity to animals, of course.
I think the term alignment subjectivity wasn’t used before, looks like a useful term, let’s coin it :)