I think in a practical computer setting this would be resolved by making a design choice how the environment is modeled. That is there is an implicit assumtion that for any “reasonable” caringness it must be expressible in the vocabulary we set up for the system. If the system doesn’t track salt levels then it’s not a valid care target. One might think just doing ontology doesn’t involve making preference choice but making some preferences impossible to articulate it is in fact a partial preference choice.
For actual human we build our world understanding from meaningful objects. Thus the feeling of “what is” accompanies “what is meanigful”. If it kicks like a duck and sounds like a duck it is a duck and not a secret combination of possibly being a mix of bucket or duck. If the bucket was secretly radioactive and a human solved the problem by refilling the bucket one would blame ignorance and not knowing what was important (it’s more reasonable for a human to taste a salt level differnce, it’s more plausible to say “I couldn’t know” about radioactivity).
One might think just doing ontology doesn’t involve making preference choice but making some preferences impossible to articulate it is in fact a partial preference choice.
Yep, that’s my argument: some (but not all) aspects of human preferences have to be included in the setup somehow.
it’s more reasonable for a human to taste a salt level differnce, it’s more plausible to say “I couldn’t know” about radioactivity
I hope you don’t taste every bucket of water before putting it away! ^_^
I think in a practical computer setting this would be resolved by making a design choice how the environment is modeled. That is there is an implicit assumtion that for any “reasonable” caringness it must be expressible in the vocabulary we set up for the system. If the system doesn’t track salt levels then it’s not a valid care target. One might think just doing ontology doesn’t involve making preference choice but making some preferences impossible to articulate it is in fact a partial preference choice.
For actual human we build our world understanding from meaningful objects. Thus the feeling of “what is” accompanies “what is meanigful”. If it kicks like a duck and sounds like a duck it is a duck and not a secret combination of possibly being a mix of bucket or duck. If the bucket was secretly radioactive and a human solved the problem by refilling the bucket one would blame ignorance and not knowing what was important (it’s more reasonable for a human to taste a salt level differnce, it’s more plausible to say “I couldn’t know” about radioactivity).
Yep, that’s my argument: some (but not all) aspects of human preferences have to be included in the setup somehow.
I hope you don’t taste every bucket of water before putting it away! ^_^