I mostly agree with Zack_M_Davis that this is a solved problem, although rather than talking about a formalization of causality I’d say this is a special case of epistemic circularity and thus an instance of the problem of the criterion. There’s nothing unusual going on with counterfactuals other than that people sometimes get confused about what propositions are (e.g. they believe propositions have some sort of absolute truth beyond causality because they fail to realize epistemology is grounded in purpose rather than something eternal and external to the physical world) and then go on to get mixed up into thinking that something special must be going on with counterfactuals due to their confusion about propositions in general.
I don’t know if I’ll personally get around to explaining this in more detail, but I think this is low hanging fruit since it falls out so readily from understanding the contingency of epistemology caused by the problem of the criterion.
I think A is solved, though I wouldn’t exactly phrase it like that, more like counterfactuals make sense because they are what they are and knowledge works the way it does.
Zack seems to be making a claim to B, but I’m not expert enough in decision theory to say much about it.
I don’t think they’re really at odds. Zack’s analysis cuts off at a point where the circularity exists below it. There’s still the standard epistemic circularity that exists whenever you try to ground out any proposition, counterfactual or not, but there’s a level of abstraction where you can remove the seeming circularity by shoving it lower or deeper into the reduction of the proposition towards grounding out in some experience.
Another way to put this is that we can choose what to be pragmatic about. Zack’s analysis choosing to be pragmatic about counterfactuals at the level of making decisions, and this allows removing the circularity up to the purpose of making a decision. If we want to be pragmatic about, say, accurately predicting what we will observe about the world, then there’s still some weird circularity in counterfactuals to be addressed if we try to ask questions like “why these counterfactuals rather than others?” or “why can we formulate counterfactuals at all?”.
Also I guess I should be clear that there’s no circularity outside the map. Circularity is entirely a feature of our models of reality rather than reality itself. That’s way, for example, the analysis on epistemic circularity I offer is that we can ground things out in purpose and thus the circularity was actually an illusion of trying to ground truth in itself rather than experience.
I’m not sure I’ve made this point very clearly elsewhere before, so sorry if that’s a bit confusing. The point is that circularity is a feature of the relative rather than the absolute, so circularity exists in the map but not the territory. We only get circularity by introducing abstractions that can allow things in the map to depend on each other rather than the territory.
I wouldn’t be surprised if other concepts such as probability were circular in the same way as counterfactuals, although I feel that this is more than just a special case of epistemic circularity. Like I agree that we can only reason starting from where we are—rather than from the view from nowhere—but counterfactuals feel different because they are such a fundamental concept that appears everywhere. As an example, our understanding of chairs doesn’t seem circular in quite the same sense. That said, I’d love to see someone explore this line of thought.
Zack’s analysis cuts off at a point where the circularity exists below it
I could be wrong, but I suspect Zack would disagree with the notion that there is a circularity below it involving counterfactuals. I wouldn’t be surprised though if Zack acknowledge a circularity not involving counterfactuals.
Also I guess I should be clear that there’s no circularity outside the map. Circularity is entirely a feature of our models of reality rather than reality itself
Agreed. That said, I don’t think counterfactuals are in the territory. I think I said before that they were in the map, although I’m now leaning away from that characterisation as I feel that they are more of a fundamental category that we use to draw the map.
Agreed. That said, I don’t think counterfactuals are in the territory. I think I said before that they were in the map, although I’m now leaning away from that characterisation as I feel that they are more of a fundamental category that we use to draw the map.
Yes, I think there is something interesting going on where human brains seem to operate in a way that makes counterfactuals natural. I actually don’t think there’s anything special about counterfactuals, though, just that the human brain is designed such that thoughts are not strongly tethered to sensory input vs. “memory” (internally generated experience), but that’s perhaps only subtly different than saying counterfactuals rather than something powering them is a fundamental feature of how our minds work.
I think I disagree here. I’m working on an entry to OP’s competition which will contain an argument showing some inherent convergence between different agent’s counterfactuals, due to the structure of the universe.
I think this is just agreement then? That minds are influenced by the structure of the universe they operate in in similar ways sounds like exactly what we should expect. That doesn’t mean we need to elevate such convergence to be something more than intersubjective agreement about reality.
Causation is a feature of models, not reality. We need only suppose reality is one thing after another (or not even that! reality is just this moment, which for us contains a sensation we call a memory of past moments), and any causal structure is inferred to exist rather than something we directly observe. I make this argument in some detail here: https://www.lesswrong.com/posts/RMBMf85gGYytvYGBv/no-causation-without-reification
I agree that causal structure is inferred to exist, and never directly observable. However, the universe has certain properties that makes it very hard not to infer a causal structure if we want to model it, in particular:
A constant increase in entropy
Deterministic laws relating the past and the future
… which have symmetry across time and space
It seems exponentially hard to account for this without causality.
Formally, we often model causation as the action of one thing implying another, and we might formalize this with mathematical notation like A⟹B to mean some event or thing A causes some other event or thing B to happen.
I immediately disagree here, formally we usually model causality as our observations being generated by some sort of dynamical system. This cannot be specified with a mathematical notation like implication.
Try to imagine what the world is like if you’re not modeling it. Are you picturing atoms? Particles? Wave functions? Strings?
Wrong!
Those are all models we impose on the world to make sense of it. Useful ones, usually, but still models.
Sure, I know, but that doesn’t mean there’s no dynamical process generating the territory, only that we don’t know which one (and maybe can’t know).
The territory, noting that the notion that such a thing exists is itself another model, is a kind of soup of stuff that’s all mixed up and irreducible. You either have everything or nothing, for all time or no time. If you want anything else, you have to draw a map, make distinctions, and make choices.
Those distinctions are hopefully correlated with how the territory works, because that makes them useful for things like predicting what will happen (and we often call this correspondence “truth”), but they are not the territory itself. The territory just is; always has been, always will be.
This means that there’s no aspect of the territory that is causality. There’s no A, there’s no B, there’s no ⟹, there’s just “is”.
A and B are typically high-level features in our models that simplify the territory; as a result, the causality in our models will also be simplifications of the causality in the territory.
But without causality, I don’t see how you’d get thermodynamics. That seems like a “just is” that is best accounted for causally, even if we don’t have the exact causal theory underlying it. (Somehow, thermodynamics has managed to hold even as we’ve repeatedly updated our models, because it doesn’t depend on the exact causal model, but instead follows from deep aspects of the causal structure of reality.)
But if it’s not in the territory, why is causality so useful that it’s largely invisible to us, and why is it so hard for us to wrap our minds around what the world would be like if we didn’t perceive its presence?
Well, there clearly is some feature of the territory we’re carving out, making into a thing (“reifying”), and calling causality. But, we often forget, almost the instant we’ve done this, that we were the ones who did the carving, and then imagine that our perception of the world is how it really is.
This is important because it can lead us to making mistakes by thinking we understand something we don’t. We fail to notice how confused we are about the aspect of reality we call causality because it’s so close to our faces we don’t realize it’s a lens through which we are looking all the time.
But if causality is describing some feature of reality, and the feature it is describing is not itself causal, then what is the feature it is describing?
I mostly agree with Zack_M_Davis that this is a solved problem, although rather than talking about a formalization of causality I’d say this is a special case of epistemic circularity and thus an instance of the problem of the criterion. There’s nothing unusual going on with counterfactuals other than that people sometimes get confused about what propositions are (e.g. they believe propositions have some sort of absolute truth beyond causality because they fail to realize epistemology is grounded in purpose rather than something eternal and external to the physical world) and then go on to get mixed up into thinking that something special must be going on with counterfactuals due to their confusion about propositions in general.
I don’t know if I’ll personally get around to explaining this in more detail, but I think this is low hanging fruit since it falls out so readily from understanding the contingency of epistemology caused by the problem of the criterion.
Which part are you claiming is a solved problem? Is it:
a) That counterfactuals can only be understood within the counterfactual perspective OR
b) The implications of this for decision theory OR
c) Both
I think A is solved, though I wouldn’t exactly phrase it like that, more like counterfactuals make sense because they are what they are and knowledge works the way it does.
Zack seems to be making a claim to B, but I’m not expert enough in decision theory to say much about it.
Sorry, when you say A is solved, you’re claiming that the circularity is known to be true, right?
Zack seems to be claiming that Bayesian Networks both draw out the implications and show that the circularity is false.
So unless I’m misunderstanding you, your answer seems to be at odds with Zack.
I don’t think they’re really at odds. Zack’s analysis cuts off at a point where the circularity exists below it. There’s still the standard epistemic circularity that exists whenever you try to ground out any proposition, counterfactual or not, but there’s a level of abstraction where you can remove the seeming circularity by shoving it lower or deeper into the reduction of the proposition towards grounding out in some experience.
Another way to put this is that we can choose what to be pragmatic about. Zack’s analysis choosing to be pragmatic about counterfactuals at the level of making decisions, and this allows removing the circularity up to the purpose of making a decision. If we want to be pragmatic about, say, accurately predicting what we will observe about the world, then there’s still some weird circularity in counterfactuals to be addressed if we try to ask questions like “why these counterfactuals rather than others?” or “why can we formulate counterfactuals at all?”.
Also I guess I should be clear that there’s no circularity outside the map. Circularity is entirely a feature of our models of reality rather than reality itself. That’s way, for example, the analysis on epistemic circularity I offer is that we can ground things out in purpose and thus the circularity was actually an illusion of trying to ground truth in itself rather than experience.
I’m not sure I’ve made this point very clearly elsewhere before, so sorry if that’s a bit confusing. The point is that circularity is a feature of the relative rather than the absolute, so circularity exists in the map but not the territory. We only get circularity by introducing abstractions that can allow things in the map to depend on each other rather than the territory.
I wouldn’t be surprised if other concepts such as probability were circular in the same way as counterfactuals, although I feel that this is more than just a special case of epistemic circularity. Like I agree that we can only reason starting from where we are—rather than from the view from nowhere—but counterfactuals feel different because they are such a fundamental concept that appears everywhere. As an example, our understanding of chairs doesn’t seem circular in quite the same sense. That said, I’d love to see someone explore this line of thought.
I could be wrong, but I suspect Zack would disagree with the notion that there is a circularity below it involving counterfactuals. I wouldn’t be surprised though if Zack acknowledge a circularity not involving counterfactuals.
Agreed. That said, I don’t think counterfactuals are in the territory. I think I said before that they were in the map, although I’m now leaning away from that characterisation as I feel that they are more of a fundamental category that we use to draw the map.
Yes, I think there is something interesting going on where human brains seem to operate in a way that makes counterfactuals natural. I actually don’t think there’s anything special about counterfactuals, though, just that the human brain is designed such that thoughts are not strongly tethered to sensory input vs. “memory” (internally generated experience), but that’s perhaps only subtly different than saying counterfactuals rather than something powering them is a fundamental feature of how our minds work.
I think I disagree here. I’m working on an entry to OP’s competition which will contain an argument showing some inherent convergence between different agent’s counterfactuals, due to the structure of the universe.
I think this is just agreement then? That minds are influenced by the structure of the universe they operate in in similar ways sounds like exactly what we should expect. That doesn’t mean we need to elevate such convergence to be something more than intersubjective agreement about reality.
If minds are influenced by the structure of the universe, then that requires some causal structure of the universe to influence them.
Causation is a feature of models, not reality. We need only suppose reality is one thing after another (or not even that! reality is just this moment, which for us contains a sensation we call a memory of past moments), and any causal structure is inferred to exist rather than something we directly observe. I make this argument in some detail here: https://www.lesswrong.com/posts/RMBMf85gGYytvYGBv/no-causation-without-reification
I feel a bit confused.
I agree that causal structure is inferred to exist, and never directly observable. However, the universe has certain properties that makes it very hard not to infer a causal structure if we want to model it, in particular:
A constant increase in entropy
Deterministic laws relating the past and the future
… which have symmetry across time and space
It seems exponentially hard to account for this without causality.
When opening the post:
I immediately disagree here, formally we usually model causality as our observations being generated by some sort of dynamical system. This cannot be specified with a mathematical notation like implication.
Sure, I know, but that doesn’t mean there’s no dynamical process generating the territory, only that we don’t know which one (and maybe can’t know).
A and B are typically high-level features in our models that simplify the territory; as a result, the causality in our models will also be simplifications of the causality in the territory.
But without causality, I don’t see how you’d get thermodynamics. That seems like a “just is” that is best accounted for causally, even if we don’t have the exact causal theory underlying it. (Somehow, thermodynamics has managed to hold even as we’ve repeatedly updated our models, because it doesn’t depend on the exact causal model, but instead follows from deep aspects of the causal structure of reality.)
But if causality is describing some feature of reality, and the feature it is describing is not itself causal, then what is the feature it is describing?