Fair enough. And I certainly agree that there is a lot of bathwater! The bundle of connotations attached to the word “wanting” is a mess. I just want to flag that it seems to me that much of the normal ontology can be rescued, albeit with a little bit of work. I claim that concepts like corrigibility are still useful and coherent once the rescuing has taken place.
I think the ontology Steven describes makes a lot of sense for an organism in any environment that contain rocks and plants (easy to predict), weather patterns and running water (hard to predict short term, Vingean approaches of identifying their goals are not very helpful), and other humans and sentient animals (hard to predict short term, but Vingean approaches of identifying their goals are very helpful). This is a good heuristic for an evolved animal to recognize other evolved animals. So of course we have special-purpose learning modules designed for it, and can make this judgement without effort. In fact, since it’s an important part of our threat monitoring systems, our special-purpose abilities in this area are tuned towards avoiding false negatives and tolerating some false positives.
Fair enough. And I certainly agree that there is a lot of bathwater! The bundle of connotations attached to the word “wanting” is a mess. I just want to flag that it seems to me that much of the normal ontology can be rescued, albeit with a little bit of work. I claim that concepts like corrigibility are still useful and coherent once the rescuing has taken place.
I think the ontology Steven describes makes a lot of sense for an organism in any environment that contain rocks and plants (easy to predict), weather patterns and running water (hard to predict short term, Vingean approaches of identifying their goals are not very helpful), and other humans and sentient animals (hard to predict short term, but Vingean approaches of identifying their goals are very helpful). This is a good heuristic for an evolved animal to recognize other evolved animals. So of course we have special-purpose learning modules designed for it, and can make this judgement without effort. In fact, since it’s an important part of our threat monitoring systems, our special-purpose abilities in this area are tuned towards avoiding false negatives and tolerating some false positives.