Elucidating the genetic architecture of human values would be enormously useful for AI alignment, in my opinion.
After reading this post and the responses, I feel like a lot of the apparent disagreement boils down to confusion around terms with only a very little actual disagreement about strength of certain forces in relation to other forces.
I wanted to highlight this important source of agreement, rather than focus on the disagreement. I would hope that everyone who agrees with this post and/or with shard theory as it has so far been developed would be enthusiastically on board with this key point. I believe there is substantial value to alignment research in understanding the genetic architecture of human values.
After reading this post and the responses, I feel like a lot of the apparent disagreement boils down to confusion around terms with only a very little actual disagreement about strength of certain forces in relation to other forces.
I wanted to highlight this important source of agreement, rather than focus on the disagreement. I would hope that everyone who agrees with this post and/or with shard theory as it has so far been developed would be enthusiastically on board with this key point. I believe there is substantial value to alignment research in understanding the genetic architecture of human values.