There are two issues here: we don’t know what to want (or what to care about), and even if we did, we don’t know how to evaluate the world from that perspective (e.g. if conscious beings, or beings capable of pain, are what we should care about—we do not presently know which possible entities would be conscious, and which ones not).
It is sometimes suggested that pain and pleasure are the only true ends-in-themselves, and would form the basis of a natural morality. In that case, we should be prioritizing that aspect of consciousness studies. There are, roughly speaking, two relevant directions of inquiry. One is refining the human understanding of consciousness in itself, the other is understanding how consciousness fits into the scientific worldview.
Regarding the first direction, the formulation of concepts like qualia, intentionality, and the unity of consciousness (not to mention many more arcane concepts from phenomenology) represent advancements in the clarity and correctness with which we can think about consciousness. Regarding the second direction, this is roughly the problem of how mind and matter are related, and the creation of an ontology which actually includes both.
The situation suggests to me that we should be prioritizing progress (from those two perspectives) regarding the nature of pleasure and pain, since this will increase the chance that whatever value systems and ontologies govern the first superintelligence(s) are on the right track.
There are two issues here: we don’t know what to want (or what to care about), and even if we did, we don’t know how to evaluate the world from that perspective (e.g. if conscious beings, or beings capable of pain, are what we should care about—we do not presently know which possible entities would be conscious, and which ones not).
It is sometimes suggested that pain and pleasure are the only true ends-in-themselves, and would form the basis of a natural morality. In that case, we should be prioritizing that aspect of consciousness studies. There are, roughly speaking, two relevant directions of inquiry. One is refining the human understanding of consciousness in itself, the other is understanding how consciousness fits into the scientific worldview.
Regarding the first direction, the formulation of concepts like qualia, intentionality, and the unity of consciousness (not to mention many more arcane concepts from phenomenology) represent advancements in the clarity and correctness with which we can think about consciousness. Regarding the second direction, this is roughly the problem of how mind and matter are related, and the creation of an ontology which actually includes both.
The situation suggests to me that we should be prioritizing progress (from those two perspectives) regarding the nature of pleasure and pain, since this will increase the chance that whatever value systems and ontologies govern the first superintelligence(s) are on the right track.