If this were a discussion regarding, say, “artificial geologists”, and the question was “Why do artificial geologists usually have a concept of ‘rock’?”, it would not be regarded as very mysterious. A geologist needs the idea of a rock because rocks exist, and are basic to their domain of expertise. The general principle might be, “if it exists and is relevant to the agent, the agent will have a representation of it”.
I take it that the major puzzle here is that counterfactuals are situations which do not exist, and yet CSAs contain representations of them. That’s “could” (and “would” is a form of “could”); “should” seems less of a problem, since while the counterfactuals don’t exist, the representations of them do, and so a decision process in which such representations contribute to the decision does not in itself involve anything like nonexistent entities having a causal effect on existent entities.
So the basic problem remains: Why do CSAs (a felicitous naming, by the way) contain representations of things which do not exist?
I would think the basic answer is that reality and possibility are combinatorial and lawful in the same way. The situations which actually occur, and the situations which could have occurred, are made of the same sort of entities (combinatorial in the same way), and the way to reason validly about their properties is also the same (lawful in the same way). The difference between a representation of an existing situation and a representation of a merely possible situation is not intrinsically fundamental—it is contingent on external circumstances, namely, what actually happens in the world. As a matter of computer science or cognitive science, there is no significant formal difference between isa(president,obama) and isa(president,mccain), even though only one of them represents a reality.
To sum up, I think that the answers to all your other questions will be technical unfoldings of the original insight that computationally, there is no significant difference between a representation of a counterfactual and a representation of a reality.
If this were a discussion regarding, say, “artificial geologists”, and the question was “Why do artificial geologists usually have a concept of ‘rock’?”, it would not be regarded as very mysterious. A geologist needs the idea of a rock because rocks exist, and are basic to their domain of expertise. The general principle might be, “if it exists and is relevant to the agent, the agent will have a representation of it”.
I take it that the major puzzle here is that counterfactuals are situations which do not exist, and yet CSAs contain representations of them. That’s “could” (and “would” is a form of “could”); “should” seems less of a problem, since while the counterfactuals don’t exist, the representations of them do, and so a decision process in which such representations contribute to the decision does not in itself involve anything like nonexistent entities having a causal effect on existent entities.
So the basic problem remains: Why do CSAs (a felicitous naming, by the way) contain representations of things which do not exist?
I would think the basic answer is that reality and possibility are combinatorial and lawful in the same way. The situations which actually occur, and the situations which could have occurred, are made of the same sort of entities (combinatorial in the same way), and the way to reason validly about their properties is also the same (lawful in the same way). The difference between a representation of an existing situation and a representation of a merely possible situation is not intrinsically fundamental—it is contingent on external circumstances, namely, what actually happens in the world. As a matter of computer science or cognitive science, there is no significant formal difference between isa(president,obama) and isa(president,mccain), even though only one of them represents a reality.
To sum up, I think that the answers to all your other questions will be technical unfoldings of the original insight that computationally, there is no significant difference between a representation of a counterfactual and a representation of a reality.