Are you trying to work on a thing that can discern the principles and implications of objective benevolence, if such principles exist? (If the question seems unclear: I mean this in a manner similar to how mathematicians can discern and explicitly and rigorously discuss the structure of Peano arithmetic.)
Relatedly: do you agree that the idea of “Alignment” is kind of a mediocre way to think about not getting killed by AGI, and might even be evil, because an “aligned AI” could be “aligned with the Devil” or “aligned with a psychotic dictator” or “aligned with… <anything, really>”?
In a deep sense: please talk about “how you think about” whether or how the most plausible versions of “moral realism” are, or are not, relevant to your project.
This is a genuinely difficult and interesting question that I want to provide a good answer for, but that might take me some time to write up, I’ll get back to you at a later date.
I worked as an algorithmic ethicist for a blockchain project for several years, and this was (arguably?) my central professional bedevilment. It doesn’t really surprise me that you have a hard time with it… I asked it because it is The Tough One, and if you had an actually good answer then such an answer would (probably) count as “non-trivial research progress”.
Are you trying to work on a thing that can discern the principles and implications of objective benevolence, if such principles exist? (If the question seems unclear: I mean this in a manner similar to how mathematicians can discern and explicitly and rigorously discuss the structure of Peano arithmetic.)
Relatedly: do you agree that the idea of “Alignment” is kind of a mediocre way to think about not getting killed by AGI, and might even be evil, because an “aligned AI” could be “aligned with the Devil” or “aligned with a psychotic dictator” or “aligned with… <anything, really>”?
In a deep sense: please talk about “how you think about” whether or how the most plausible versions of “moral realism” are, or are not, relevant to your project.
This is a genuinely difficult and interesting question that I want to provide a good answer for, but that might take me some time to write up, I’ll get back to you at a later date.
I like that you didn’t say something glib :-)
I worked as an algorithmic ethicist for a blockchain project for several years, and this was (arguably?) my central professional bedevilment. It doesn’t really surprise me that you have a hard time with it… I asked it because it is The Tough One, and if you had an actually good answer then such an answer would (probably) count as “non-trivial research progress”.