Some senses of “erroneous” that might be involved here include (this list is not necessarily intended to be exhaustive):
Mathematically incorrect—i.e. the proofs contain actual logical inconsistencies. This was argued by some early skeptics (such as Kronecker) but is basically indefensible ever since the formulation of axiomatic set theory and results such as Gödel’s on the consistency of the Axiom of Choice. Such a person would have to actually believe the ZF axioms are inconsistent, and I am aware of no plausible argument for this.
Making claims that are epistemologically indefensible, even if possibly true. E.g., maybe there does exist a well-ordering of the reals, but mere mortals are in no position to assert that such a thing exists. Again, axiomatic formalization should have meant the end of this as a plausible stance.
Irrelevant or uninteresing as an area of research because of a “lack of correspondence” with “reality” or “the physical world”. In order to be consistent, a person subscribing to this view would have to repudiate the whole of pure mathematics as an enterprise. If, as is more common, the person is selectively criticizing certain parts of mathematics, then they are almost certainly suffering from map-territory confusion. Mathematics is not physics; the map is not the territory. It is not ordained or programmed into the universe that positive integers must refer specifically to numbers of elementary particles, or some such, any more than the symbolic conventions of your atlas are programmed into the Earth. Hence one cannot make a leap e.g. from the existence of a finite number of elementary particles to the theoretical adequacy of finitely many numbers. To do so would be to prematurely circumscribe the nature of mathematical models of the physical world. Any criticism of a particular area of mathematics as “unconnected to reality” necessarily has to be made from the standpoint of a particular model of reality. But part (perhaps a large part) of the point of doing pure mathematics (besides the fact that it’s fun, of course), is to prepare for the necessity, encountered time and time again in the history of our species, of upgrading—and thus changing—our very model. Not just the model itself but the ways in which mathematical ideas are used in the model. This has often happened in ways that (at least at the time) would have seemed very surprising.
For the sake of argument, I will go ahead and ask what sort of nonconstructive entities you think an AI needs to reason about, in order to function properly.
Well, if the AI is doing mathematics, then it needs to reason about the very same entities that human mathematicians reason about.
Maybe that sounds like begging the question, because you could ask why humans themselves need to reason about those entities (which is kind of the whole point here). But in that case I’m not sure what you’re getting at by switching from humans to AIs.
Do you perhaps mean to ask something like: “What kind of mathematical entities will be needed in order to formulate the most fundamental physical laws?”
Why do you think that the axiomatic formulation of ZFC “should have meant an end” to the stance that ZFC makes claims that are epistemologically indefensible? Just because I can formalize a statement does not make that statement true, even if it is consistent. Many people (including me and apparently Eliezer, though I would guess that my views are different from his) do not think that the axioms of ZFC are self-evident truths.
In general, I find the argument for Platonism/the validity of ZFC based on common acceptance to be problematic because I just don’t think that most people think about these issues seriously. It is a consensus of convenience and inertia. Also, many mathematicians are not Platonists at all but rather formalists—and constructivism is closer to formalism than Platonism is.
It’s rude to start refuting an idea before you’ve finished defining it.
One of these things is not like the others. There’s nothing wrong with giving us a history of constructive thinking, and providing us with reasons why outdated versions of the theory were found wanting. It’s good style to use parallel construction to build rhetorical momentum. It is terribly dishonest to do both at the same time—it creates the impression that the subjective reasons you give for dismissing point 3 have weight equal to the objective reasons history has given for dismissing points 1 and 2.
Your talk in point 3 about “map-territory confusion” is very strange. Mathematics is all in your head. It’s all map, no territory. You seem to be claiming that constructivsts are outside of the mathematical mainstream because they want to bend theory in the direction of a preferred outcome. You then claim that this is outside of the bounds of acceptable mathematical thinking, So what’s wrong with reasoning like this:
“Nobody really likes all of the consequences of the Axiom of Choice, but most people seem willing to put up with its bad behavior because some of the abstractions it enables—like the Real Numbers—are just so damn useful. I wonder how many of the useful properties of the Real Numbers I could capture by building up from (a possibly weakened version of) ZF set theory and a weakened version of the Axiom of Choice?”
I’m sorry, but I don’t think there was anything remotely “rude” or “terribly dishonest” about my previous comment. If you think I am mistaken about anything I said, just explain why. Criticizing my rhetorical style and accusing me of violating social norms is not something I find helpful.
Quite frankly, I also find criticisms of the form “you sound more confident than you should be” rather annoying. E.g:
it creates the impression that the subjective reasons you give for dismissing point 3 have weight equal to the objective reasons history has given for dismissing points 1 and 2.
That’s because for me, the reasons I gave in point 3 do indeed have similar weight to the reasons I gave in points 1 and 2. If you disagree, by all means say so. But to rise up in indignation over the very listing of my reasons—is that really necessary? Would you seriously have preferred that I just list the bullet points without explaining what I thought?
So what’s wrong with reasoning like this:
Nothing at all, except for the false claim that nobody likes the consequences of the Axiom of Choice. (Some people do like them, and why shouldn’t they?)
The target of my critique—and I thought I made this clear in my response to cousin_it—is the critique of mainstream mathematical reasoning, not the research program of exploring different axiomatic set theories. The latter could easily be done by someone fully on board with the soundness of traditional mathematics. Just as it is unnecessary to doubt the correctness of Euclid’s arguments in order to be interested in non-Euclidean geometry.
Criticizing my rhetorical style and accusing me of violating social norms is not something I find helpful.
Until very recently, I held a similar attitude. I think it’s common to be annoyed by this sort of criticism… it’s distracting and rarely relevant.
That said, it seems to me that the above “rarely” isn’t rare enough. If you’re inadvertently violating a social norm, wouldn’t you like to know? If you already know, what does it matter to have it pointed out to you? Just ignore the redundant information.
I think this principle extends to a lot of speculative or subjective criticism. The potential value of just one accurate critique taken to heart seems quite high. Does such criticism have a positive expected value? That depends on the overall cost of the associated inaccurate or redundant statements (i.e., the vast majority of them). It seems this cost can be made to approach zero by just not taking them personally and ignoring them when they’re misguided, so long as they’re sufficiently disentangled from “object-level” statements.
Some senses of “erroneous” that might be involved here include (this list is not necessarily intended to be exhaustive):
Mathematically incorrect—i.e. the proofs contain actual logical inconsistencies. This was argued by some early skeptics (such as Kronecker) but is basically indefensible ever since the formulation of axiomatic set theory and results such as Gödel’s on the consistency of the Axiom of Choice. Such a person would have to actually believe the ZF axioms are inconsistent, and I am aware of no plausible argument for this.
Making claims that are epistemologically indefensible, even if possibly true. E.g., maybe there does exist a well-ordering of the reals, but mere mortals are in no position to assert that such a thing exists. Again, axiomatic formalization should have meant the end of this as a plausible stance.
Irrelevant or uninteresing as an area of research because of a “lack of correspondence” with “reality” or “the physical world”. In order to be consistent, a person subscribing to this view would have to repudiate the whole of pure mathematics as an enterprise. If, as is more common, the person is selectively criticizing certain parts of mathematics, then they are almost certainly suffering from map-territory confusion. Mathematics is not physics; the map is not the territory. It is not ordained or programmed into the universe that positive integers must refer specifically to numbers of elementary particles, or some such, any more than the symbolic conventions of your atlas are programmed into the Earth. Hence one cannot make a leap e.g. from the existence of a finite number of elementary particles to the theoretical adequacy of finitely many numbers. To do so would be to prematurely circumscribe the nature of mathematical models of the physical world. Any criticism of a particular area of mathematics as “unconnected to reality” necessarily has to be made from the standpoint of a particular model of reality. But part (perhaps a large part) of the point of doing pure mathematics (besides the fact that it’s fun, of course), is to prepare for the necessity, encountered time and time again in the history of our species, of upgrading—and thus changing—our very model. Not just the model itself but the ways in which mathematical ideas are used in the model. This has often happened in ways that (at least at the time) would have seemed very surprising.
Well, if the AI is doing mathematics, then it needs to reason about the very same entities that human mathematicians reason about.
Maybe that sounds like begging the question, because you could ask why humans themselves need to reason about those entities (which is kind of the whole point here). But in that case I’m not sure what you’re getting at by switching from humans to AIs.
Do you perhaps mean to ask something like: “What kind of mathematical entities will be needed in order to formulate the most fundamental physical laws?”
Why do you think that the axiomatic formulation of ZFC “should have meant an end” to the stance that ZFC makes claims that are epistemologically indefensible? Just because I can formalize a statement does not make that statement true, even if it is consistent. Many people (including me and apparently Eliezer, though I would guess that my views are different from his) do not think that the axioms of ZFC are self-evident truths.
In general, I find the argument for Platonism/the validity of ZFC based on common acceptance to be problematic because I just don’t think that most people think about these issues seriously. It is a consensus of convenience and inertia. Also, many mathematicians are not Platonists at all but rather formalists—and constructivism is closer to formalism than Platonism is.
Regarding your three bullet points above:
It’s rude to start refuting an idea before you’ve finished defining it.
One of these things is not like the others. There’s nothing wrong with giving us a history of constructive thinking, and providing us with reasons why outdated versions of the theory were found wanting. It’s good style to use parallel construction to build rhetorical momentum. It is terribly dishonest to do both at the same time—it creates the impression that the subjective reasons you give for dismissing point 3 have weight equal to the objective reasons history has given for dismissing points 1 and 2.
Your talk in point 3 about “map-territory confusion” is very strange. Mathematics is all in your head. It’s all map, no territory. You seem to be claiming that constructivsts are outside of the mathematical mainstream because they want to bend theory in the direction of a preferred outcome. You then claim that this is outside of the bounds of acceptable mathematical thinking, So what’s wrong with reasoning like this:
“Nobody really likes all of the consequences of the Axiom of Choice, but most people seem willing to put up with its bad behavior because some of the abstractions it enables—like the Real Numbers—are just so damn useful. I wonder how many of the useful properties of the Real Numbers I could capture by building up from (a possibly weakened version of) ZF set theory and a weakened version of the Axiom of Choice?”
I’m sorry, but I don’t think there was anything remotely “rude” or “terribly dishonest” about my previous comment. If you think I am mistaken about anything I said, just explain why. Criticizing my rhetorical style and accusing me of violating social norms is not something I find helpful.
Quite frankly, I also find criticisms of the form “you sound more confident than you should be” rather annoying. E.g:
That’s because for me, the reasons I gave in point 3 do indeed have similar weight to the reasons I gave in points 1 and 2. If you disagree, by all means say so. But to rise up in indignation over the very listing of my reasons—is that really necessary? Would you seriously have preferred that I just list the bullet points without explaining what I thought?
Nothing at all, except for the false claim that nobody likes the consequences of the Axiom of Choice. (Some people do like them, and why shouldn’t they?)
The target of my critique—and I thought I made this clear in my response to cousin_it—is the critique of mainstream mathematical reasoning, not the research program of exploring different axiomatic set theories. The latter could easily be done by someone fully on board with the soundness of traditional mathematics. Just as it is unnecessary to doubt the correctness of Euclid’s arguments in order to be interested in non-Euclidean geometry.
Until very recently, I held a similar attitude. I think it’s common to be annoyed by this sort of criticism… it’s distracting and rarely relevant.
That said, it seems to me that the above “rarely” isn’t rare enough. If you’re inadvertently violating a social norm, wouldn’t you like to know? If you already know, what does it matter to have it pointed out to you? Just ignore the redundant information.
I think this principle extends to a lot of speculative or subjective criticism. The potential value of just one accurate critique taken to heart seems quite high. Does such criticism have a positive expected value? That depends on the overall cost of the associated inaccurate or redundant statements (i.e., the vast majority of them). It seems this cost can be made to approach zero by just not taking them personally and ignoring them when they’re misguided, so long as they’re sufficiently disentangled from “object-level” statements.