If you appeal to intuitions about rigor, it’s not so much an outlier since fear and excitement must be aspects of rigorously reconstructed preference as well.
I find myself simultaneously convinced and unconvinced by this! Anticipation (dependent, of course, on your definition) is surely a vital tool in any agent that wants to steer the future? Or do you mean ‘human anticipation’ as differentiated from other kinds? In which case, what demarcates that from whatever an AI would do in thinking about the future?
However, Dai, your top level comment sums up my eventual thoughts on this problem very well. I’ve been trying for a long time to resign myself to the idea that a notion of discrete personal experience is incompatible with what we know about the world. Doesn’t make it any easier though.
My two cents—the answer to this trilemma will come from thinking about the system as a whole rather than personal experience. Can we taboo ‘personal experience’ and find a less anthropocentric way to think about this?
This presumably places anticipation together with excitement and fear—an aspect of human experience, but not a useful concept for decision theory.
I’m not convinced that “It turns out that pi is in fact greater than three” is a mere aspect of human experience.
If you appeal to intuitions about rigor, it’s not so much an outlier since fear and excitement must be aspects of rigorously reconstructed preference as well.
I find myself simultaneously convinced and unconvinced by this! Anticipation (dependent, of course, on your definition) is surely a vital tool in any agent that wants to steer the future? Or do you mean ‘human anticipation’ as differentiated from other kinds? In which case, what demarcates that from whatever an AI would do in thinking about the future?
However, Dai, your top level comment sums up my eventual thoughts on this problem very well. I’ve been trying for a long time to resign myself to the idea that a notion of discrete personal experience is incompatible with what we know about the world. Doesn’t make it any easier though.
My two cents—the answer to this trilemma will come from thinking about the system as a whole rather than personal experience. Can we taboo ‘personal experience’ and find a less anthropocentric way to think about this?