I don’t really disagree with the main claim here, but I’ll steelman the opposite claim for a moment. Why call the problem of the criterion open?
To my knowledge (and please tell me if I’m wrong here), there is no widely accepted mathematical framework for the problem of the criterion in which the problem has been proved unsolvable. In that regard it is not analogous to e.g. Gödel’s theorems. This is important: if some formal version of the problem of the criterion comes up when I’m working on a theorem about agency, or trying to design an AI architecture with some property, then I want the formal argument, not just a natural-language argument that my problem is intractable. Such natural-language arguments are not particularly reliable; they tend to sneak in a bunch of hidden premises, and a mathematical version of the problem which shows up in practice can violate those hidden premises.
For example: for most of the 20th century, it was basically-universally accepted that no statistical analysis of correlation could reliably establish causation. Post-Judea-Pearl, this is clearly wrong. The formal arguments that correlation cannot establish causation had loopholes in them—most importantly, they were only about two variables, and largely fell apart with three or more variables. If I were working on some theorem about AI or agency, and wanted to show something about an agent’s ability to deduce causation from observation of a large number of variables, I might have noticed my inability to prove the theorem I wanted. At the very least, I would have noticed the lack of a robust mathematical framework for talking about what causality even is, and likely would have needed to develop one. (Indeed, this is basically what Pearl and others did.) But the natural language arguments glossed over such subtleties; it wasn’t until people actually started developing the mathematical framework for talking about causality that we noticed correlative data could be sufficient to deduce it.
By contrast, I find it hard to imagine something like that being overlooked by Gödel’s theorems. There, we do have a mathematical framework, and we know what kinds-of-things allow loopholes, and roughly how big those loopholes can be.
I don’t see any framework for the problem of the criterion which would make me confident that we won’t have a repeat of “correlation doesn’t imply causation”, the way Gödel’s theorems give me such confidence. Again, this may just be my ignorance in not having read up on the topic much; please correct me if so.
I’m not aware of anything quite so rigorous beyond what we might call “philosophical math” of using words in a precise way to evaluate doxastic logic. Maybe this is enough, but does feel like we should at least write it down somewhere in formal notation to make sure there’s no gaps.
I’m not sure what you would write down (see my comment on the ambiguities of your statement of the problem). But here is one possible version.
Suppose that there’s some set of sentences S (finite, or infinite). We can “believe” some subset of these sentences, B. (For probabilists, we can include statements-of-probability as sentences, EG “it will rain today with p < .4″.)
A justification function J is supposed to map each belief b to a justifying set, J(b). J(b) should be a subset of B (we should believe our justification). The idea is that the sentences in J(b), taken together, should provide a sufficient justification in the belief b.
Think of J(b) as defining a directed graph on B, where x->y if x is in J(y). Now, some assumptions:
There should be no root notes, meaning x where J(x) is the empty set. This codifies the intuition that every belief should have a justification.
Justification should be well-founded: every non-empty subset C of B should have a least element, IE an x such that there is no y∈C such that y->x. This captures the intuition that infinite chains of justification don’t count, including circular reasoning that goes around forever and ever, and also non-repeating infinite chains where you can justify each statement with some other, but “nothing justifies the whole chain”.
Theorem: There is no suitable notion of justification J() which matches the above axioms for non-empty belief sets B.
Proof: Let C=B. Axiom 1 says that there are no root nodes. Since this choice of C is nonempty, Axiom 2 requires that there exist some root node.
As a mathematical theorem, this is not very interesting. What makes it interesting is the independent plausibility of the two axioms with respect to an intuitive notion of justified belief.
For example, formal logic does not try to support a notion of justification following the above axioms. But intuitively, the above reasoning is still behind assertions like “you need to assume something in order to prove anything”.
The fact that Chisholm’s discussion oscillates between these two versions of the Problem of the Criterion and the fact that he seems to be aware of the two versions of the problem help make it clear that perhaps there is no such thing as the Problem of the Criterion. Perhaps the Problem of the Criterion is rather a set of related problems. This is something that many philosophers since Chisholm, and Chisholm himself (see his 1977), have noted.
So I think one could just as well say that Gödel’s incompleteness theorems formalize the problem of the criterion, or perhaps even cite the Löbstacle as a formalization of the problem.
I don’t really disagree with the main claim here, but I’ll steelman the opposite claim for a moment. Why call the problem of the criterion open?
To my knowledge (and please tell me if I’m wrong here), there is no widely accepted mathematical framework for the problem of the criterion in which the problem has been proved unsolvable. In that regard it is not analogous to e.g. Gödel’s theorems. This is important: if some formal version of the problem of the criterion comes up when I’m working on a theorem about agency, or trying to design an AI architecture with some property, then I want the formal argument, not just a natural-language argument that my problem is intractable. Such natural-language arguments are not particularly reliable; they tend to sneak in a bunch of hidden premises, and a mathematical version of the problem which shows up in practice can violate those hidden premises.
For example: for most of the 20th century, it was basically-universally accepted that no statistical analysis of correlation could reliably establish causation. Post-Judea-Pearl, this is clearly wrong. The formal arguments that correlation cannot establish causation had loopholes in them—most importantly, they were only about two variables, and largely fell apart with three or more variables. If I were working on some theorem about AI or agency, and wanted to show something about an agent’s ability to deduce causation from observation of a large number of variables, I might have noticed my inability to prove the theorem I wanted. At the very least, I would have noticed the lack of a robust mathematical framework for talking about what causality even is, and likely would have needed to develop one. (Indeed, this is basically what Pearl and others did.) But the natural language arguments glossed over such subtleties; it wasn’t until people actually started developing the mathematical framework for talking about causality that we noticed correlative data could be sufficient to deduce it.
By contrast, I find it hard to imagine something like that being overlooked by Gödel’s theorems. There, we do have a mathematical framework, and we know what kinds-of-things allow loopholes, and roughly how big those loopholes can be.
I don’t see any framework for the problem of the criterion which would make me confident that we won’t have a repeat of “correlation doesn’t imply causation”, the way Gödel’s theorems give me such confidence. Again, this may just be my ignorance in not having read up on the topic much; please correct me if so.
Mathematics is not exempt from the problem of the criterion.
I’m not aware of anything quite so rigorous beyond what we might call “philosophical math” of using words in a precise way to evaluate doxastic logic. Maybe this is enough, but does feel like we should at least write it down somewhere in formal notation to make sure there’s no gaps.
I’m not sure what you would write down (see my comment on the ambiguities of your statement of the problem). But here is one possible version.
Suppose that there’s some set of sentences S (finite, or infinite). We can “believe” some subset of these sentences, B. (For probabilists, we can include statements-of-probability as sentences, EG “it will rain today with p < .4″.)
A justification function J is supposed to map each belief b to a justifying set, J(b). J(b) should be a subset of B (we should believe our justification). The idea is that the sentences in J(b), taken together, should provide a sufficient justification in the belief b.
Think of J(b) as defining a directed graph on B, where x->y if x is in J(y). Now, some assumptions:
There should be no root notes, meaning x where J(x) is the empty set. This codifies the intuition that every belief should have a justification.
Justification should be well-founded: every non-empty subset C of B should have a least element, IE an x such that there is no y∈C such that y->x. This captures the intuition that infinite chains of justification don’t count, including circular reasoning that goes around forever and ever, and also non-repeating infinite chains where you can justify each statement with some other, but “nothing justifies the whole chain”.
Theorem: There is no suitable notion of justification J() which matches the above axioms for non-empty belief sets B.
Proof: Let C=B. Axiom 1 says that there are no root nodes. Since this choice of C is nonempty, Axiom 2 requires that there exist some root node.
As a mathematical theorem, this is not very interesting. What makes it interesting is the independent plausibility of the two axioms with respect to an intuitive notion of justified belief.
For example, formal logic does not try to support a notion of justification following the above axioms. But intuitively, the above reasoning is still behind assertions like “you need to assume something in order to prove anything”.
But, really, I think the problem of the criterion is supposed to be a vague cluster and the above comes nowhere near capturing the extent of it. Internet Encyclopedia of Philosophy on the subject:
So I think one could just as well say that Gödel’s incompleteness theorems formalize the problem of the criterion, or perhaps even cite the Löbstacle as a formalization of the problem.