To comment on the object (that is: meta) level discussion: One of the most popular theories of metaphilosophy states that philosophy is “conceptual analysis”.
The obvious question is: What is “conceptual analysis”? The theory applies quite well to cases where we have general terms like “knowledge”, “probability” or “explanation”, and where we try to find definitions for them, definitions that are adequate to our antecedent intuitive understanding of those terms. What counts as a “definition”? That’s a case of conceptual analysis itself, but the usual answer is that a good definition lists individually necessary and jointly sufficient conditions. And how do we find those?
That’s where intuitions and thought experiments come in. There is a special kind of semantic intuition of the “I know it when I see it” kind. We know whether a term does or doesn’t apply to a concrete example when we see it, even if we can’t readily produce a definition. You might think being black is a necessary condition for our concept of “raven”. Now if you see something that looks like a prototypical raven, but with darkish grey feathers, would you still call it a raven? Presumably yes. You can just imagine the example of a grey raven, i.e. you do a thought experiment, and your semantic intuition tells you whether the term applies. So you discovered that being black is not a necessary condition for being a raven, and hence it isn’t part of their definition.
Similarly, you might want to know whether some properties are sufficient for applying a term, and a thought experiment could disprove this, if you imagine a thing which has all those properties but to which the term, nonetheless, doesn’t intuitively apply. (A well known example for this happened in the case of knowledge.) Note also that these (semantic) intuitions seem very reliable, in contrast to the usual “intuitive guesses”, which are simply unjustified and uncertain beliefs about factual matters, and which are also called “intuitions”.
It’s interesting that conceptual analysis bears some resemblance to the scientific method: Experiments are replaced by thought experiments, observations are replaced by semantic intuitions. Moreover, if I remember correctly, research in experimental philosophy (a branch of psychology) found that that intuitions in philosophical thought experiments are fairly stable across different people.
Now this semantic theory of metaphilosophy applies well to analyzing the meaning of general terms. It is less clear whether it applies to philosophy as a whole. For example, are philosophical ethicists really analyzing the meaning of terms like “good” and “ought”? (In fact I think so, yes.)
And which concepts do philosophical decision theorists analyze according to the semantic theory? Or logicians? Well, perhaps concepts like “rational decision” or “logical consequence”. A subject area like decision theory could include a host of interrelated concepts, which stand in certain logical relations to each other, and an axiomatization provides an implicit definition of the involved terms. Like the axioms of (second-order) Peano arithmetic seem to define, in a sense, the meaning of “natural numbers”. Another worry is that the conceptual analysis theory deals easily with “what is X?” questions (where X is “rationality”, “causation” etc.) but not so obviously with “why” questions like “why does anything exist?” or “why does consciousness exist?”. Though one could argue that the analysis of “explanation” should give some insight here. And also these questions seem not immediately semantic: “Is induction justified? If so, how?” Though again, maybe we simply still lack a general theory of induction which would help analyze the concepts of “inductive inference” and “inductive justification”, and which would answer these questions.
So the semantic theory is a bit murky outside the prototypical examples. But it seems to fit well with the “deconfunsion theory”, since conceptual analysis would aid in clarifying the relations between concepts. For example: Ask a philosophically uninformed person what the relationship is between knowledge and belief. I’ve done it several times. They struggle for confused examples, they know the difference between those terms, but only as a disposition to use them correctly when presented with specific cases, not when asked about an abstract relationship. Philosophers won’t get confused by this case (they’d say knowledge implies belief, and belief doesn’t imply knowledge), so the deconfunsion worked. They will still get confused when asked about other concepts, like when asked about the relationship between knowledge and information.
My guess is that a sufficiently general AI would be quite good at philosophy. Because deconfusing its conceptual framework has instrumental value. The danger comes from somewhat narrow superintelligences, which could have some important parts of their cognition “hard coded”, like AIXI always using the conditionalization rule. And natural selection heavily optimized animals for generality first, and increased intelligence came only slowly, while this situation seems to be reversed in AI. Highly specialized narrow systems came first. So there is some risk the first powerful superintelligence could be relatively narrow and hence have dangerous philosophical blind spots.
Thanks for this clear explanation of conceptual analysis. I’ve been wanting to ask some questions about this line of thought:
Where do semantic intuitions come from?
What should we do when different people have different such intuitions? For example you must know that Newcomb’s problem is famously divisive, with roughly half of philosophers preferring one-boxing and half preferring two-boxing. Similarly for trolley thought experiments, intuitions about the nature of morality (metaethics), etc.
How do we make sure that AI has the right intuitions? Maybe in some cases we can just have it learn from humans, but what about:
Cases where humans disagree.
Cases where all/most humans are wrong. (In other words, can we build AIs that have better intuitions than humans?) Or is that not a thing in conceptual analysis, i.e., semantic intuitions can’t be wrong?
Completely novel philosophical questions or situations where AI can’t learn from humans (because humans don’t have intuitions about it either, or AI has to make time sensitive decisions and humans are too slow).
I think concepts are probably similar to what artificial feedforward networks implement when they recognize objects. So a NN that recognizes chairs would implement the concept associated with the term “chair”. Such networks just output a value (yes/no, or something in between) when given certain, e.g. visual, inputs. Otherwise it’s a blackbox, there is no way to easily get the definition of “chair” out, even if it correctly identifies all and only chairs. And these “yes” or “no” values, when presented with specific examples as input, seem to be just what we receive from semantic intuitions. I know a chair when I see it.
Now for the practice philosophy, it is clear that we aren’t just able to apply concepts to real (e.g. sensory) data, but also to thought experiments, to hypothetical or counterfactual, in any case simulated, situations. It is not clear how this ability works in the brain, but we do have it.
When people have different intuitions in thought experiments, this could be due to several reasons:
One possibility is that the term in question is simply ambiguous. Does a tree falling in the forest make a sound when nobody is there? That presumably depends on the ambiguity of “sound”: The tree produces a sound wave, but no conscious sound experience. In such cases there is no real disagreement, just two concepts for one term.
Another possibility is that the term in question is vague. Do traffic lights have yellow or orange lights? Maybe “disagreements” here are just due to slightly different boundaries of concepts for different individuals, but there is no significant disagreement.
The last possibility is that the concepts in question are really approximately the same, and ambiguity or vagueness is not the issue. Those are typically the controversial cases. They are often called a paradox. My guess is that they are caused by some hidden complexity or ambiguity in the thought experiment or problem statement (rather than in an ambiguity of a central term) which pulls semantic intuitions in different directions. A paradox may be solved when the reasons for those contradicting intuitions are uncovered.
I actually think it is fairly rare for a paradox that some people simply have completely different intuitions. Most people can see both intuitions and are puzzled, since they (seem to) contradict each other.
In his original paper about Newcomb’s problem, I think Robert Nozick does a very good job at describing both intuitions such that both seem plausible. An example of what I imagine a solution could look like: The two-boxer answer is the right response to the question “What is the most useful decision in the given situation?”, while the one-boxer answer is the right response to the question “In the given situation, what is the decision according to the most useful general decision-making algorithm an agent could have?” Which would mean the intuitions apply to slightly different questions, even though the terms in question are not ambiguous themselves. The disagreement was semantic only insofar the problem is interpreted differently. (This is just an example of how one could, perhaps, explain the disagreement in this paradox consistent with the semantic theory, not a fleshed-out proposal.)
Ethics and so on seem similar. Generally, if a thought experiment produces very different outcomes for different people, the problem in the thought experiment my not be as clear as it seems. Maybe the problem needs clarification, or different, less unclear, thought experiments altogether.
I actually do think that semantic intuitions are infallible when they are certain. For example, if I imagine a prototypical (black) raven, and I mentally make it grey, I would still call it a raven. My semantic intuition here represents just a disposition to use the term associated with the concept. If someone then convinces me to call only black birds ravens, that wouldn’t be a counterexample to infallibility, that would just be me using a different concept than before for the same term. For paradoxical cases the intuitions are typically far less than certain, and that reflects their being provisional.
For AI to do philosophy, according to the conceptual analysis view, it needs some ability to do thought experiments, to do suppositional reasoning, and to apply its usual concepts to these virtual situations. It also needs some minimal amount of “creativity” to come up with provisional definitions or axiomatizations, and specific thought experiments. Overall, I don’t think AI would need to learn doing philosophy from humans. Either it can do it itself, possibly at a superhuman level, because it is general enough to have the necessary base abilities, or it can’t do it much at all.
To comment on the object (that is: meta) level discussion: One of the most popular theories of metaphilosophy states that philosophy is “conceptual analysis”.
The obvious question is: What is “conceptual analysis”? The theory applies quite well to cases where we have general terms like “knowledge”, “probability” or “explanation”, and where we try to find definitions for them, definitions that are adequate to our antecedent intuitive understanding of those terms. What counts as a “definition”? That’s a case of conceptual analysis itself, but the usual answer is that a good definition lists individually necessary and jointly sufficient conditions. And how do we find those?
That’s where intuitions and thought experiments come in. There is a special kind of semantic intuition of the “I know it when I see it” kind. We know whether a term does or doesn’t apply to a concrete example when we see it, even if we can’t readily produce a definition. You might think being black is a necessary condition for our concept of “raven”. Now if you see something that looks like a prototypical raven, but with darkish grey feathers, would you still call it a raven? Presumably yes. You can just imagine the example of a grey raven, i.e. you do a thought experiment, and your semantic intuition tells you whether the term applies. So you discovered that being black is not a necessary condition for being a raven, and hence it isn’t part of their definition.
Similarly, you might want to know whether some properties are sufficient for applying a term, and a thought experiment could disprove this, if you imagine a thing which has all those properties but to which the term, nonetheless, doesn’t intuitively apply. (A well known example for this happened in the case of knowledge.) Note also that these (semantic) intuitions seem very reliable, in contrast to the usual “intuitive guesses”, which are simply unjustified and uncertain beliefs about factual matters, and which are also called “intuitions”.
It’s interesting that conceptual analysis bears some resemblance to the scientific method: Experiments are replaced by thought experiments, observations are replaced by semantic intuitions. Moreover, if I remember correctly, research in experimental philosophy (a branch of psychology) found that that intuitions in philosophical thought experiments are fairly stable across different people.
Now this semantic theory of metaphilosophy applies well to analyzing the meaning of general terms. It is less clear whether it applies to philosophy as a whole. For example, are philosophical ethicists really analyzing the meaning of terms like “good” and “ought”? (In fact I think so, yes.)
And which concepts do philosophical decision theorists analyze according to the semantic theory? Or logicians? Well, perhaps concepts like “rational decision” or “logical consequence”. A subject area like decision theory could include a host of interrelated concepts, which stand in certain logical relations to each other, and an axiomatization provides an implicit definition of the involved terms. Like the axioms of (second-order) Peano arithmetic seem to define, in a sense, the meaning of “natural numbers”. Another worry is that the conceptual analysis theory deals easily with “what is X?” questions (where X is “rationality”, “causation” etc.) but not so obviously with “why” questions like “why does anything exist?” or “why does consciousness exist?”. Though one could argue that the analysis of “explanation” should give some insight here. And also these questions seem not immediately semantic: “Is induction justified? If so, how?” Though again, maybe we simply still lack a general theory of induction which would help analyze the concepts of “inductive inference” and “inductive justification”, and which would answer these questions.
So the semantic theory is a bit murky outside the prototypical examples. But it seems to fit well with the “deconfunsion theory”, since conceptual analysis would aid in clarifying the relations between concepts. For example: Ask a philosophically uninformed person what the relationship is between knowledge and belief. I’ve done it several times. They struggle for confused examples, they know the difference between those terms, but only as a disposition to use them correctly when presented with specific cases, not when asked about an abstract relationship. Philosophers won’t get confused by this case (they’d say knowledge implies belief, and belief doesn’t imply knowledge), so the deconfunsion worked. They will still get confused when asked about other concepts, like when asked about the relationship between knowledge and information.
My guess is that a sufficiently general AI would be quite good at philosophy. Because deconfusing its conceptual framework has instrumental value. The danger comes from somewhat narrow superintelligences, which could have some important parts of their cognition “hard coded”, like AIXI always using the conditionalization rule. And natural selection heavily optimized animals for generality first, and increased intelligence came only slowly, while this situation seems to be reversed in AI. Highly specialized narrow systems came first. So there is some risk the first powerful superintelligence could be relatively narrow and hence have dangerous philosophical blind spots.
Thanks for this clear explanation of conceptual analysis. I’ve been wanting to ask some questions about this line of thought:
Where do semantic intuitions come from?
What should we do when different people have different such intuitions? For example you must know that Newcomb’s problem is famously divisive, with roughly half of philosophers preferring one-boxing and half preferring two-boxing. Similarly for trolley thought experiments, intuitions about the nature of morality (metaethics), etc.
How do we make sure that AI has the right intuitions? Maybe in some cases we can just have it learn from humans, but what about:
Cases where humans disagree.
Cases where all/most humans are wrong. (In other words, can we build AIs that have better intuitions than humans?) Or is that not a thing in conceptual analysis, i.e., semantic intuitions can’t be wrong?
Completely novel philosophical questions or situations where AI can’t learn from humans (because humans don’t have intuitions about it either, or AI has to make time sensitive decisions and humans are too slow).
I think concepts are probably similar to what artificial feedforward networks implement when they recognize objects. So a NN that recognizes chairs would implement the concept associated with the term “chair”. Such networks just output a value (yes/no, or something in between) when given certain, e.g. visual, inputs. Otherwise it’s a blackbox, there is no way to easily get the definition of “chair” out, even if it correctly identifies all and only chairs. And these “yes” or “no” values, when presented with specific examples as input, seem to be just what we receive from semantic intuitions. I know a chair when I see it.
Now for the practice philosophy, it is clear that we aren’t just able to apply concepts to real (e.g. sensory) data, but also to thought experiments, to hypothetical or counterfactual, in any case simulated, situations. It is not clear how this ability works in the brain, but we do have it.
When people have different intuitions in thought experiments, this could be due to several reasons:
One possibility is that the term in question is simply ambiguous. Does a tree falling in the forest make a sound when nobody is there? That presumably depends on the ambiguity of “sound”: The tree produces a sound wave, but no conscious sound experience. In such cases there is no real disagreement, just two concepts for one term.
Another possibility is that the term in question is vague. Do traffic lights have yellow or orange lights? Maybe “disagreements” here are just due to slightly different boundaries of concepts for different individuals, but there is no significant disagreement.
The last possibility is that the concepts in question are really approximately the same, and ambiguity or vagueness is not the issue. Those are typically the controversial cases. They are often called a paradox. My guess is that they are caused by some hidden complexity or ambiguity in the thought experiment or problem statement (rather than in an ambiguity of a central term) which pulls semantic intuitions in different directions. A paradox may be solved when the reasons for those contradicting intuitions are uncovered.
I actually think it is fairly rare for a paradox that some people simply have completely different intuitions. Most people can see both intuitions and are puzzled, since they (seem to) contradict each other.
In his original paper about Newcomb’s problem, I think Robert Nozick does a very good job at describing both intuitions such that both seem plausible. An example of what I imagine a solution could look like: The two-boxer answer is the right response to the question “What is the most useful decision in the given situation?”, while the one-boxer answer is the right response to the question “In the given situation, what is the decision according to the most useful general decision-making algorithm an agent could have?” Which would mean the intuitions apply to slightly different questions, even though the terms in question are not ambiguous themselves. The disagreement was semantic only insofar the problem is interpreted differently. (This is just an example of how one could, perhaps, explain the disagreement in this paradox consistent with the semantic theory, not a fleshed-out proposal.)
Ethics and so on seem similar. Generally, if a thought experiment produces very different outcomes for different people, the problem in the thought experiment my not be as clear as it seems. Maybe the problem needs clarification, or different, less unclear, thought experiments altogether.
I actually do think that semantic intuitions are infallible when they are certain. For example, if I imagine a prototypical (black) raven, and I mentally make it grey, I would still call it a raven. My semantic intuition here represents just a disposition to use the term associated with the concept. If someone then convinces me to call only black birds ravens, that wouldn’t be a counterexample to infallibility, that would just be me using a different concept than before for the same term. For paradoxical cases the intuitions are typically far less than certain, and that reflects their being provisional.
For AI to do philosophy, according to the conceptual analysis view, it needs some ability to do thought experiments, to do suppositional reasoning, and to apply its usual concepts to these virtual situations. It also needs some minimal amount of “creativity” to come up with provisional definitions or axiomatizations, and specific thought experiments. Overall, I don’t think AI would need to learn doing philosophy from humans. Either it can do it itself, possibly at a superhuman level, because it is general enough to have the necessary base abilities, or it can’t do it much at all.