This was interesting to read but not entirely sure just what it means to me and thinking anything through.
As I was reading I started to think along the lines of you policy side—perhaps the question is not about how to twist A code into outputting Y but rather why not just consider the agent runs some other code. (The immediate problem with my thought is that of the infinite regress.) But also when thinking about counterfactuals that is in a sense what I am exploring. But I would express that more as what if action/path Y were taken, where does that lead? Is that a better result? If so then the response is about updating some priors related to the input to A or updating A. In this sense the question is not about the logical problems of getting A to output Y when we know it has to output X but about the internal logic and decision making performance of A and if we need to update to A’.
I am also wondering if including the whole free-will aspect add value. If you just took that aspect out what changes for your thinking? Or is the whole question of free-will part of the philosophical question you need to address. If so your post did prompt a thought on my thinking about free-will, particularly in the context of rational mindsets. I don’t know if someone has already followed the line of thinking (but would certainly think so) but I don’t think free-will can be rationally explored and explained within the confines of pure logical consistency.
Without some assumption similar to “free will” it is hard to do any decision theory at all, as you can’t compare different actions; there is only one possible action.
The counterfactual nonrealist position is closer to determinism than the policy-dependent source code position. This assumes that the algorithm controls the decision while the output of the algorithm is unknown.
Without some assumption similar to “free will” it is hard to do any decision theory at all, as you can’t compare different actions; there is only one possible action.
Under determinism, there is only one actually possible action, and that doesn’t stop you comparing hypothetical actions. Logical possibility =/= real possibility. Since logical possibilities are only logical possibilities, no sincere assumption of real free will is required.
Since you are invariably in a far from omniscient state about both the world and your own inner workings, you are pretty much always dealing with hypotheses, not direct insight into reality.
Under determinism, you should be a nonrealist about real counterfactuals, but there is still no problem with logical counterfactuals. So what is “the problem of logical counterfactuals”?
They’re logically incoherent so your reasoning about them is limited. If you gain in computing power then you need to stop being a realist about them or else your reasoning explodes.
They are not logically incoherent in thenselves. They are inconsistent with what actually happened. That means that if you try to be bundle the hypothetical,the logical counterfactual ,in with your model of reality, the resulting mish mash will be inconsistent. But the resulting mish mash isn’t the logical counterfactual per se.
W can think about counterfactuals without our heads the exploding. That is the correct starting point. How is that possible? The obvious answer is that consideration of hypothetical scenarios takes place in a sandbox.
They are logically incoherent in themselves though. Suppose the agent’s source code is “A”. Suppose that in fact, A returns action X. Consider a logical counterfactual “possible world” where A returns action Y. In this logical counterfactual, it is possible to deduce a contradiction: A returns X (by computation/logic) and returns Y (by assumption) and X is not equal to Y. Hence by the principle of explosion, everything is true.
It isn’t necessary to observe that A returns X in real life, it can be deduced from logic.
(Note that this doesn’t exclude the logical material conditionals described in the post, only logical counterfactuals)
Source code doesn’t entirely determine the result, inputs are also required.* Thus “logical counterfactuals” -reasoning about what a program will return if I input y? This can be done by asking ‘if I had input y instead of x’ or ‘if I input y’ even if I later decide to input x.
While it can be said that such considerations render one’s “output” conditional on logic, they remain entirely conditional on reasoning about a model, which may be incorrect. It seems more useful to refer to such a relation as conditional on one’s models/reasoning, or even processes in the world. A calculator may be misused—a 2 instead of a 3 here, hitting “=” one too many times, there, etc.
(Saying it is impossible for a rational agent that knows X to do Y, and agent A is not doing Y, does not establish that A is irrational—even if the premises are true, what follows is that A is not rational or does not know X.)
*Unless source code is defined as including the inputs.
You are assuming a very strong set of conditions..that determinism holds,that the agent has perfect knowledge of its source code, and that it is compelled to consider hypothetical situations in maximum resolution.
Those are the conditions in which logical counterfactuals are most well-motivated. If there isn’t determinism or known source code then there isn’t an obvious reason to be considering impossible possible worlds.
Those are the conditions under which counterfactuals are flat out impossible. But we have plenty of motivation to consider hypotheticals ,and we don’t generally know how possible they are
This was interesting to read but not entirely sure just what it means to me and thinking anything through.
As I was reading I started to think along the lines of you policy side—perhaps the question is not about how to twist A code into outputting Y but rather why not just consider the agent runs some other code. (The immediate problem with my thought is that of the infinite regress.) But also when thinking about counterfactuals that is in a sense what I am exploring. But I would express that more as what if action/path Y were taken, where does that lead? Is that a better result? If so then the response is about updating some priors related to the input to A or updating A. In this sense the question is not about the logical problems of getting A to output Y when we know it has to output X but about the internal logic and decision making performance of A and if we need to update to A’.
I am also wondering if including the whole free-will aspect add value. If you just took that aspect out what changes for your thinking? Or is the whole question of free-will part of the philosophical question you need to address. If so your post did prompt a thought on my thinking about free-will, particularly in the context of rational mindsets. I don’t know if someone has already followed the line of thinking (but would certainly think so) but I don’t think free-will can be rationally explored and explained within the confines of pure logical consistency.
Without some assumption similar to “free will” it is hard to do any decision theory at all, as you can’t compare different actions; there is only one possible action.
The counterfactual nonrealist position is closer to determinism than the policy-dependent source code position. This assumes that the algorithm controls the decision while the output of the algorithm is unknown.
Under determinism, there is only one actually possible action, and that doesn’t stop you comparing hypothetical actions. Logical possibility =/= real possibility. Since logical possibilities are only logical possibilities, no sincere assumption of real free will is required.
Since you are invariably in a far from omniscient state about both the world and your own inner workings, you are pretty much always dealing with hypotheses, not direct insight into reality.
This is exactly what is described in the counterfactual nonrealism section.
Under determinism, you should be a nonrealist about real counterfactuals, but there is still no problem with logical counterfactuals. So what is “the problem of logical counterfactuals”?
They’re logically incoherent so your reasoning about them is limited. If you gain in computing power then you need to stop being a realist about them or else your reasoning explodes.
They are not logically incoherent in thenselves. They are inconsistent with what actually happened. That means that if you try to be bundle the hypothetical,the logical counterfactual ,in with your model of reality, the resulting mish mash will be inconsistent. But the resulting mish mash isn’t the logical counterfactual per se.
W can think about counterfactuals without our heads the exploding. That is the correct starting point. How is that possible? The obvious answer is that consideration of hypothetical scenarios takes place in a sandbox.
They are logically incoherent in themselves though. Suppose the agent’s source code is “A”. Suppose that in fact, A returns action X. Consider a logical counterfactual “possible world” where A returns action Y. In this logical counterfactual, it is possible to deduce a contradiction: A returns X (by computation/logic) and returns Y (by assumption) and X is not equal to Y. Hence by the principle of explosion, everything is true.
It isn’t necessary to observe that A returns X in real life, it can be deduced from logic.
(Note that this doesn’t exclude the logical material conditionals described in the post, only logical counterfactuals)
Source code doesn’t entirely determine the result, inputs are also required.* Thus “logical counterfactuals” -reasoning about what a program will return if I input y? This can be done by asking ‘if I had input y instead of x’ or ‘if I input y’ even if I later decide to input x.
While it can be said that such considerations render one’s “output” conditional on logic, they remain entirely conditional on reasoning about a model, which may be incorrect. It seems more useful to refer to such a relation as conditional on one’s models/reasoning, or even processes in the world. A calculator may be misused—a 2 instead of a 3 here, hitting “=” one too many times, there, etc.
(Saying it is impossible for a rational agent that knows X to do Y, and agent A is not doing Y, does not establish that A is irrational—even if the premises are true, what follows is that A is not rational or does not know X.)
*Unless source code is defined as including the inputs.
You are assuming a very strong set of conditions..that determinism holds,that the agent has perfect knowledge of its source code, and that it is compelled to consider hypothetical situations in maximum resolution.
Those are the conditions in which logical counterfactuals are most well-motivated. If there isn’t determinism or known source code then there isn’t an obvious reason to be considering impossible possible worlds.
Those are the conditions under which counterfactuals are flat out impossible. But we have plenty of motivation to consider hypotheticals ,and we don’t generally know how possible they are