>Sometimes we treat our own feelings as intrinsic properties of things out there in the world—Arthur is handsome, Birthdays are exciting, Capitalism is bad, Diapers are gross, etc. (ISM §3.3.2, see also Yudkowsky 2008). When we apply that general principle to “the feeling of being surprised”, we get an intuition that objects can be intrinsically unpredictable. This intuitive concept is what I call vitalistic force (ISM §3.3), and we apply it to animals, people, cartoon characters, and machines that “seem alive” (as opposed to seeming “inanimate”). It doesn’t veridically correspond to anything in the real world (ISM §3.3.3). It amounts to a sense that something has intrinsic important unpredictability in its behaviour
Your have a section talking about intrinsic surprisingness as something that could not possibly exst in the territory. (You say that we attribute vitalistic force to things, and that vitalistic force has no characteristic other than intrinsic surprisingness) It was pointed out before, that intrinsic surprisingness is just indeterminism, and indeterminism can exist in the territory—it’s a scientifically respectable idea, and it’s not even an entity, just the relative absence of something. If vitalistic force has some further characteristics, that would make it something like a soul or elan vital, there would be a real problem .. but you don’t say that it does.
The footnote:-
>Granted, one can argue that observer-independent intrinsic unpredictability does in fact exist “in the territory”. For example, there’s a meaningful distinction between “true” quantum randomness versus pseudorandomness. However, that property in the “territory” has so little correlation with “vitalistic force” in the map, that we should really think of them as two unrelated things. For example, in my own brain, the sporadic clicking of a geiger counter feels like an inanimate (vitalistic-force-free) “mechanism”, but is intrinsically unpredictable (from a physics perspective). Conversely, a rerun cartoon depiction of Homer Simpson feels like it’s infused with vitalistic force, but in fact it’s intrinsically fully predictable. Recall also that vitalistic force is related not only to unpredictability but also physiological arousal.
..tells us that there is something additional to surprisingness but not what it is, or why it can’t exist in the territory.
>More precisely: If there are deterministic upstream explanations of what the Active Self is doing and why, e.g. via algorithmic or other mechanisms happening under the hood, then that feels like a complete undermining of one’s free will and agency
It needn’t feel like a complete undermining, because both will and agency can be cashed out in a deterministic framework. Specifically, compatibilist free will , and agency as von Neumann rationalism (about which a word of two has been written in this site).
An algorithm need not be deterministic. An organism making decisions according to an indeterministic algorithm can be seen as having libertarian free will and intrinsic unpredictability. There is therefore no gap between the algorithmic (etc) cluster of concepts, and the intrinsic unpredictablity (etc) cluster. Neither picture allows you to make exact predictions, and both allow you to use approximate heuristics … so where is the difference in practice? And if there is no difference in practice , what is so terrible about the intuitive concept of agency?
>Our intuitive notion of being manipulated is related to a person (call him Ahmed) taking an action A with the property that, in our intuitive causal world-models, the chain-of-causation leading to A does not ultimately trace back to the acausal force of Ahmed’s free will, but rather to the free will of some third-party who manipulated Ahmed.
Maybe. But what’s the problem? If B manipulates A , that means A is doing something against A’s wishes (otherwise it’s mutually beneficial) , and in favour of B.s. wishes. That account is neutral between a mechanistic ontology , where wishes are preference functions, and an intuitive ontology. Wishes, beliefs and desires are high level approximations that can be cashed out in either ontology.
>For the kind of complex algorithm that underlies human decision-making, the only way to reliably figure out what this algorithm will output, is to actually run the algorithm step-by-step and see.[9]
The only way to do better than heuristics is to run recapturing step by step .. so long as it’s actually deterministic. But we don’t know that people run off algorithms, we don’t know that the algorithms are deterministic, that’s a further assumption, and we don’t know what the algorithm is.
We are not forced into accepting the deterministic+ algorithmic picture by adopting naturalism,.since other naturalistic ontologies are available. We are also not forced to reject every aspect of the intuituve, since some can be implemented naturalistically.
Nothing practical can be done with the algorithm ontology, so.It can’t be making a difference in practice, so there can’t be a practical drawback to using the theoretically “wrong” intuitive concepts. When we as predictinf other humans, we are dealing with a largely unknown black ( or dark grey) box either way. The algorithmic approach might allow more *knowability*, but that’s an issue for the future.
What’s the actual problem?. Maybe someone thinks an LLM has vitalistic force and intrinsic surprisingness? Well, you can correct them by showing them the algorithm. You don’t have to bring in the unproven claim that humans have an algorithm.
>So it’s kinda misleading to say “the laws of physics predict that the person will decide to take action A1”. A better description is: “the laws of physics predict that the person will think it over, ponder their goals and preferences and whims, and then make a decision, and it turns out the decision they made is … [and then we pause and watch as the person takes action A1] … well, I guess it was Action A1”. And if the person had decided A2, then evidently A2 would have been the prediction of the laws of physics! In other words, if the laws of physics were deterministic, that determinism would run through the “execution of free will”; it wouldn’t circumvent it
Well, that’s a reinvention of compatibilism.
>By contrast, as in §3.3.2 above, if you look at a clockwork contraption that you don’t understand, you’ll be surprised by its behavioral outputs, but your intuitive model will say that there’s no “surprisingness” / “vitalistic force” metaphysical paint within the clockwork contraption itself. Instead, the surprise is explained away (in your intuitive model) as arising from your unfamiliarity with the details of the contraption
You are not looking at clockwork, twice over. You don’t know that people are deterministic, and don’t know exact mechanism, either. Since indeterminism is possible, intrinsic surprisingness is possible. even if souls and vital sparks arent
Your have a section talking about intrinsic surprisingness as something that could not possibly exst in the territory. It was pointed out before, that intrinsic surprisingness is just indeterminism, and indeterminism can exist in the territory—it’s a scientifically respectable idea, and it’s not even an entity, just the relative absence of something.
My longer discussion in the original does actually discuss this point in a footnote:
Granted, one can argue that observer-independent intrinsic unpredictability does in fact exist “in the territory”. For example, there’s a meaningful distinction between “true” quantum randomness versus pseudorandomness. However, that property in the “territory” has so little correlation with “vitalistic force” in the map, that we should really think of them as two unrelated things. For example, in my own brain, the sporadic clicking of a geiger counter feels like an inanimate (vitalistic-force-free) “mechanism”, but is intrinsically unpredictable (from a physics perspective). Conversely, a rerun cartoon depiction of Homer Simpson feels like it’s infused with vitalistic force, but in fact it’s intrinsically fully predictable. Recall also that vitalistic force is related not only to unpredictability but also physiological arousal.
>So it’s kinda misleading to say “the laws of physics predict that the person will decide to take action A1”. A better description is: “the laws of physics predict that the person will think it over, ponder their goals and preferences and whims, and then make a decision, and it turns out the decision they made is … [and then we pause and watch as the person takes action A1] … well, I guess it was Action A1”. And if the person had decided A2, then evidently A2 would have been the prediction of the laws of physics! In other words, if the laws of physics were deterministic, that determinism would run through the “execution of free will”; it wouldn’t circumvent it
Well, that’s a reinvention of compatibilism.
I certainly didn’t mean to suggest that that was an original take. Indeed, I’ve seen variations of that take in so many places that I wouldn’t even know who to cite. I have now added a footnote making that explicit, thanks.
>Sometimes we treat our own feelings as intrinsic properties of things out there in the world—Arthur is handsome, Birthdays are exciting, Capitalism is bad, Diapers are gross, etc. (ISM §3.3.2, see also Yudkowsky 2008). When we apply that general principle to “the feeling of being surprised”, we get an intuition that objects can be intrinsically unpredictable. This intuitive concept is what I call vitalistic force (ISM §3.3), and we apply it to animals, people, cartoon characters, and machines that “seem alive” (as opposed to seeming “inanimate”). It doesn’t veridically correspond to anything in the real world (ISM §3.3.3). It amounts to a sense that something has intrinsic important unpredictability in its behaviour
Your have a section talking about intrinsic surprisingness as something that could not possibly exst in the territory. (You say that we attribute vitalistic force to things, and that vitalistic force has no characteristic other than intrinsic surprisingness) It was pointed out before, that intrinsic surprisingness is just indeterminism, and indeterminism can exist in the territory—it’s a scientifically respectable idea, and it’s not even an entity, just the relative absence of something. If vitalistic force has some further characteristics, that would make it something like a soul or elan vital, there would be a real problem .. but you don’t say that it does.
The footnote:-
>Granted, one can argue that observer-independent intrinsic unpredictability does in fact exist “in the territory”. For example, there’s a meaningful distinction between “true” quantum randomness versus pseudorandomness. However, that property in the “territory” has so little correlation with “vitalistic force” in the map, that we should really think of them as two unrelated things. For example, in my own brain, the sporadic clicking of a geiger counter feels like an inanimate (vitalistic-force-free) “mechanism”, but is intrinsically unpredictable (from a physics perspective). Conversely, a rerun cartoon depiction of Homer Simpson feels like it’s infused with vitalistic force, but in fact it’s intrinsically fully predictable. Recall also that vitalistic force is related not only to unpredictability but also physiological arousal.
..tells us that there is something additional to surprisingness but not what it is, or why it can’t exist in the territory.
>More precisely: If there are deterministic upstream explanations of what the Active Self is doing and why, e.g. via algorithmic or other mechanisms happening under the hood, then that feels like a complete undermining of one’s free will and agency
It needn’t feel like a complete undermining, because both will and agency can be cashed out in a deterministic framework. Specifically, compatibilist free will , and agency as von Neumann rationalism (about which a word of two has been written in this site).
An algorithm need not be deterministic. An organism making decisions according to an indeterministic algorithm can be seen as having libertarian free will and intrinsic unpredictability. There is therefore no gap between the algorithmic (etc) cluster of concepts, and the intrinsic unpredictablity (etc) cluster. Neither picture allows you to make exact predictions, and both allow you to use approximate heuristics … so where is the difference in practice? And if there is no difference in practice , what is so terrible about the intuitive concept of agency?
>Our intuitive notion of being manipulated is related to a person (call him Ahmed) taking an action A with the property that, in our intuitive causal world-models, the chain-of-causation leading to A does not ultimately trace back to the acausal force of Ahmed’s free will, but rather to the free will of some third-party who manipulated Ahmed.
Maybe. But what’s the problem? If B manipulates A , that means A is doing something against A’s wishes (otherwise it’s mutually beneficial) , and in favour of B.s. wishes. That account is neutral between a mechanistic ontology , where wishes are preference functions, and an intuitive ontology. Wishes, beliefs and desires are high level approximations that can be cashed out in either ontology.
>For the kind of complex algorithm that underlies human decision-making, the only way to reliably figure out what this algorithm will output, is to actually run the algorithm step-by-step and see.[9]
The only way to do better than heuristics is to run recapturing step by step .. so long as it’s actually deterministic. But we don’t know that people run off algorithms, we don’t know that the algorithms are deterministic, that’s a further assumption, and we don’t know what the algorithm is.
We are not forced into accepting the deterministic+ algorithmic picture by adopting naturalism,.since other naturalistic ontologies are available. We are also not forced to reject every aspect of the intuituve, since some can be implemented naturalistically.
Nothing practical can be done with the algorithm ontology, so.It can’t be making a difference in practice, so there can’t be a practical drawback to using the theoretically “wrong” intuitive concepts. When we as predictinf other humans, we are dealing with a largely unknown black ( or dark grey) box either way. The algorithmic approach might allow more *knowability*, but that’s an issue for the future.
What’s the actual problem?. Maybe someone thinks an LLM has vitalistic force and intrinsic surprisingness? Well, you can correct them by showing them the algorithm. You don’t have to bring in the unproven claim that humans have an algorithm.
>So it’s kinda misleading to say “the laws of physics predict that the person will decide to take action A1”. A better description is: “the laws of physics predict that the person will think it over, ponder their goals and preferences and whims, and then make a decision, and it turns out the decision they made is … [and then we pause and watch as the person takes action A1] … well, I guess it was Action A1”. And if the person had decided A2, then evidently A2 would have been the prediction of the laws of physics! In other words, if the laws of physics were deterministic, that determinism would run through the “execution of free will”; it wouldn’t circumvent it
Well, that’s a reinvention of compatibilism.
>By contrast, as in §3.3.2 above, if you look at a clockwork contraption that you don’t understand, you’ll be surprised by its behavioral outputs, but your intuitive model will say that there’s no “surprisingness” / “vitalistic force” metaphysical paint within the clockwork contraption itself. Instead, the surprise is explained away (in your intuitive model) as arising from your unfamiliarity with the details of the contraption
You are not looking at clockwork, twice over. You don’t know that people are deterministic, and don’t know exact mechanism, either. Since indeterminism is possible, intrinsic surprisingness is possible. even if souls and vital sparks arent
My longer discussion in the original does actually discuss this point in a footnote:
I certainly didn’t mean to suggest that that was an original take. Indeed, I’ve seen variations of that take in so many places that I wouldn’t even know who to cite. I have now added a footnote making that explicit, thanks.