There are two important inputs in any mathematical AI alignment which need to be provided by humans or taken as un-said assumptions: who are “humans” and what is “human values” – or normatively important part of human values. AI can guess it, bit it needs AI Psychology and AI Sociology, which also could be biased.
Also see my short-form today about about reflexive stability of AI alignment.
“Scale-free, physical” ethics (notably, interacting with naturalistic theories of consciousness rather than phenomenological, hermeneutic, and other “continental” studies and accounts of consciousness)
Clearly, technical AI alignment cannot take a “human” with some formulaic definition as only or primary moral subjects and/or subjects to be aligned with. This itself would be unscientific. Rather, alignment should be based on some naturalistic theory of ethics, e.g., saying that moral subjectivity is proportional to integrated informationΦ in the agent’s consciousness. The “values” are also determined scientifically, from the game-theoretic/evolutionary setup.
So, a cyborg will also be a subject of alignment. But it also extends moral/alignment subjectivity to animals, of course.
I think the term alignment subjectivity wasn’t used before, looks like a useful term, let’s coin it :)
There are two important inputs in any mathematical AI alignment which need to be provided by humans or taken as un-said assumptions: who are “humans” and what is “human values” – or normatively important part of human values. AI can guess it, bit it needs AI Psychology and AI Sociology, which also could be biased.
Also see my short-form today about about reflexive stability of AI alignment.
It’s somewhere between the lines here:
Clearly, technical AI alignment cannot take a “human” with some formulaic definition as only or primary moral subjects and/or subjects to be aligned with. This itself would be unscientific. Rather, alignment should be based on some naturalistic theory of ethics, e.g., saying that moral subjectivity is proportional to integrated information Φ in the agent’s consciousness. The “values” are also determined scientifically, from the game-theoretic/evolutionary setup.
So, a cyborg will also be a subject of alignment. But it also extends moral/alignment subjectivity to animals, of course.
I think the term alignment subjectivity wasn’t used before, looks like a useful term, let’s coin it :)