I don’t think planaria have values, whether you view that truth as a “cop-out” or not. Even if we replace your example with the ‘minimal’ nervous system capable of having qualia—supposing the organism in question doesn’t also have speech in the usual sense—I still think that’s a terrible analogy. The reason humans can’t understand worms’ philosophies of value is because there aren’t any. The reason we can’t understand what planaria say about their values is that they can’t talk, not because they’re alien. When we put our minds to understanding an animal like a cat which evolved for (some) social interaction, we can do so—I taught a cat to signal hunger by jumping up on a particular surface, and Buddhist monks with lots of time have taught cats many more tricks. People are currently teaching them to hold English conversations (apparently) by pushing buttons which trigger voice recordings. Unsurprisingly, it looks like cats value outcomes like food in their mouths and a lack of irritating noises, not some alien goal that Stephen Hawking could never understand.
If you think that a superhuman AGI would have a lot of trouble inferring your desires or those of others, even given the knowledge it should rapidly develop about evolution—congratulations, you’re autistic.
I don’t think planaria have values, whether you view that truth as a “cop-out” or not. Even if we replace your example with the ‘minimal’ nervous system capable of having qualia—supposing the organism in question doesn’t also have speech in the usual sense—I still think that’s a terrible analogy. The reason humans can’t understand worms’ philosophies of value is because there aren’t any. The reason we can’t understand what planaria say about their values is that they can’t talk, not because they’re alien. When we put our minds to understanding an animal like a cat which evolved for (some) social interaction, we can do so—I taught a cat to signal hunger by jumping up on a particular surface, and Buddhist monks with lots of time have taught cats many more tricks. People are currently teaching them to hold English conversations (apparently) by pushing buttons which trigger voice recordings. Unsurprisingly, it looks like cats value outcomes like food in their mouths and a lack of irritating noises, not some alien goal that Stephen Hawking could never understand.
If you think that a superhuman AGI would have a lot of trouble inferring your desires or those of others, even given the knowledge it should rapidly develop about evolution—congratulations, you’re autistic.