This was an excellent post, thanks for writing it!
But, I think you unfairly dismiss the obvious solution to this madness, and I completely understand why, because it’s not at all intuitive where the problem in the setup of infinite ethics is. It’s in your choice of proof system and interpretation of mathematics! (Don’t use non-constructive proof systems!)
This is a bit of an esoteric point and I’ve been planning to write a post or even sequence about this for a while, so I won’t be able to lay out the full arguments in one comment, but let me try to convey the gist (apologies to any mathematicians reading this and spotting stupid mistakes I made):
Joe, I don’t like funky science or funky decision theory. And fair enough. But like a good Bayesian, you’ve got non-zero credence on them both (otherwise, you rule out ever getting evidence for them), and especially on the funky science one. And as I’ll discuss below, non-zero credence is enough.
This is where things go wrong. The actual credence of seeing a hypercomputer is zero, because a computationally bounded observer can never observe such an object in such a way that differentiates it from a finite approximation. As such, you should indeed have a zero percent probability of ever moving into a state in which you have performed such a verification, it is a logical impossibility. Think about what it would mean for you, a computationally bounded approximate bayesian, to come into a state of belief that you are in possession of a hypercomputer (and not a finite approximation of a hypercomputer, which is just a normal computer. Remember arbitrarily large numbers are still infinitely far away from infinity!). What evidence would you have to observe for this belief? You would need to observe literally infinite bits, and your credence to observing infinite bits should be zero, because you are computationally bounded! If you yourself are not a hypercomputer, you can never move into the state of believing a hypercomputer exists.
This is somewhat analogous to how Solomonoff inductors cannot model a universe containing themselves. Solomonoff inductors are “one step up in the halting hierarchy” from us and cannot model universes that have “super-infinite objects” like themselves in it. Similarly, we cannot model universes that contain “merely infinite” objects (and by transitivity, any super-infinite objects either) in it, either, our bayesian reasoning does not allow it!
I think the core of the problem is that, unfortunately, modern mathematics implicitly accepts classical logic as its basis of formalization, which is a problem because the Law of Excluded Middle is an implicit halting oracle. The LEM says that every logical statement is either true or false. This makes intuitive sense, but is wrong. If you think of logical statements as programs whose truth value we want to evaluate by executing a proof search, there are, in fact three “truth values”: True, false and uncomputable! This is a necessity because any axiom system worth its salt is Turing complete (this is basically what Gödel showed in his incompleteness theorems, he used Gödel numbers because Turing machines didn’t exist yet to formalize the same idea) and therefor has programs that don’t halt. Intuitionistic Logic (the logic we tend to formalize type theory and computer science with) doesn’t have this problem of an implicit halting oracle, and in my humble opinion should be used for the formalization of mathematics, on peril of trading infinite universes for an avocado sandwich and a big lizard if we use classical logic.
My own take, though, is that resting the viability of your ethics on something like “infinities aren’t a thing” is a dicey game indeed, especially given that modern cosmology says that our actual concrete universe is very plausibly infinite
Note that us using constructivist/intuitionistic logic does not mean that “infinities aren’t a thing”, it’s a bit more subtle than that (and something I have admittedly not fully deconfused for myself yet). But basically, the kind of “infinities” that cosmologists talk about are (in my ontology) very different from the “super-infinities” that you get in the limit of hypercomputation. Intuitively, it’s important to differentiate “inductive infinities” (“you need arbitrarily many steps to complete this computation”) and “real infinities” (“the solution only pops out after infinity steps have been complete” i.e. a halting oracle).
The difference makes the most sense from the perspective of computational complexity theory. The universe is a “program” of complexity class PTIME/BQP (BQP is basically just the quantum version of PTIME), which means that you can evaluate the “next state” of the universe with at most PTIME/BQP computation. Importantly, this means that even if the universe is inflationary and “infinite”, you could evaluate the state of any part of it in (arbitrarily large) finite time. There are no “effects that emerge only at infinity”. The (evaluation of a given arbitrary state of the) universe halts. This is very different to the kinds of computations a hypercomputer is capable of (and less paradoxical). Which is why I found the following very amusing:
baby-universes/wormholes/hyper-computers etc appear much more credible, at least, than “consciousness = cheesy-bread.”
Quite the opposite! Or rather, one of those three things is not like the other. baby-universes are in P/BQP, wormholes are in PSPACE (assuming by wormholes you mean closed timelike curves, which is afaik the default interpretation), and hyper-computers are halting-complete which is ludicrously insanely not even remotely like the other two things. So in that regard, yes, I think consciousness being equal to cheesy-bread is more likely than finding a hypercomputer!
To be clear when I talk about “non-constructive logic is Bad™” I don’t mean that the actual literal symbolic mathematics is somehow malign (of course), it’s the interpretation we assign to it. We think we’re reasoning about infinite objects, but we’re really reasoning about computable weaker versions of the objects, and these are not the same thing. If one is maximally careful with ones interpretations, this is (theoretically) not a problem, but this is such a subtle difference of interpretation that this is very difficult to disentangle in our mere human minds. I think this is at the heart of the problems with infinite ethics, because understanding what the correct mathematical interpretations are is so damn subtle and confusing, we find ourselves in bizarre scenarios that seem contradictory and insane because we accidentally naively extrapolate interpretations to objects they don’t belong to.
I didn’t do the best of jobs formally arguing for my point, and I’m honestly still 20% confused about this all (at least), but I hope I at least gave some interesting intuitions about why the problem might be in our philosophy of mathematics, not our philosophy of ethics.
P.S. I’m sure you’ve heard of it before, but on the off chance you haven’t, I can not recommend this wonderful paper by Scott Aaronson highly enough for a crash course in many of these kinds of topics relevant to philosophers.
The case for observing a hypercomputer might rather be that a claim that has infinidesimal credence requires infinite amounts of proof to get to a finite credence level. So a being that can only entertain finite evidence would treat that credence as effectively zero but it might technically be separate from zero.
I could imagine programming a hypertask into an object and finding some exotic trajectory with proper time more than a finite amount and receive the object from such trajectory having completed the task. The hypothesis that it was actually a very potent classical computer is ruled out by the structure of the task. I am not convinced that the main or only method of checking for nature of computation is to check output bit by bit.
This is where things go wrong. The actual credence of seeing a hypercomputer is zero, because a computationally bounded observer can never observe such an object in such a way that differentiates it from a finite approximation
This seems dubious. Compare: “the actual credence that the universe contains more computing power than my brain is zero, because an observer with the computing power of my brain can never observe such an object in such a way that differentiates it from a brain-sized approximation”. It’s true that a bounded approximation to Solomonoff induction would think this way, but that seems like a problem with Solomonoff induction, not a guide for the way we should reason ourselves. See also the discussion here on forms of hypercomputation that could be falsified in principle.
This is where things go wrong. The actual credence of seeing a hypercomputer is zero, because a computationally bounded observer can never observe such an object in such a way that differentiates it from a finite approximation. As such, you should indeed have a zero percent probability of ever moving into a state in which you have performed such a verification, it is a logical impossibility. Think about what it would mean for you, a computationally bounded approximate bayesian, to come into a state of belief that you are in possession of a hypercomputer (and not a finite approximation of a hypercomputer, which is just a normal computer. Remember arbitrarily large numbers are still infinitely far away from infinity!). What evidence would you have to observe for this belief? You would need to observe literally infinite bits, and your credence to observing infinite bits should be zero, because you are computationally bounded! If you yourself are not a hypercomputer, you can never move into the state of believing a hypercomputer exists.
Sorry, I previously assigned hypercomputers a non-zero credence, and you’re asking me to assign it zero credence. This requires an infinite amount of bits to update, which is impossible to collect in my computationally bounded state. Your case sounds sensible, but I literally can’t receive enough evidence over the course of a lifetime to be convinced by it.
Like, intuitively, it doesn’t feel literally impossible that humanity discovers a computationally unbounded process in our universe. If a convincing story is fed into my brain, with scientific consensus, personally verifying the math proof, concrete experiments indicating positive results, etc., I expect I would believe it. In my state of ignorance, I would not be surprised to find out there’s a calculation which requires a computationally unbounded process to calculate but a bounded process to verify.
To actually intuitively give something 0 (or 1) credence, though, to be so confident in a thesis that you literally can’t change your mind, that at the very least seems very weird. Self-referentially, I won’t actually assign that situation 0 credence, but even if I’m very confident that 0 credence is correct, my actual credence will be bounded by my uncertainty in my method of calculating credence.
This was an excellent post, thanks for writing it!
But, I think you unfairly dismiss the obvious solution to this madness, and I completely understand why, because it’s not at all intuitive where the problem in the setup of infinite ethics is. It’s in your choice of proof system and interpretation of mathematics! (Don’t use non-constructive proof systems!)
This is a bit of an esoteric point and I’ve been planning to write a post or even sequence about this for a while, so I won’t be able to lay out the full arguments in one comment, but let me try to convey the gist (apologies to any mathematicians reading this and spotting stupid mistakes I made):
This is where things go wrong. The actual credence of seeing a hypercomputer is zero, because a computationally bounded observer can never observe such an object in such a way that differentiates it from a finite approximation. As such, you should indeed have a zero percent probability of ever moving into a state in which you have performed such a verification, it is a logical impossibility. Think about what it would mean for you, a computationally bounded approximate bayesian, to come into a state of belief that you are in possession of a hypercomputer (and not a finite approximation of a hypercomputer, which is just a normal computer. Remember arbitrarily large numbers are still infinitely far away from infinity!). What evidence would you have to observe for this belief? You would need to observe literally infinite bits, and your credence to observing infinite bits should be zero, because you are computationally bounded! If you yourself are not a hypercomputer, you can never move into the state of believing a hypercomputer exists.
This is somewhat analogous to how Solomonoff inductors cannot model a universe containing themselves. Solomonoff inductors are “one step up in the halting hierarchy” from us and cannot model universes that have “super-infinite objects” like themselves in it. Similarly, we cannot model universes that contain “merely infinite” objects (and by transitivity, any super-infinite objects either) in it, either, our bayesian reasoning does not allow it!
I think the core of the problem is that, unfortunately, modern mathematics implicitly accepts classical logic as its basis of formalization, which is a problem because the Law of Excluded Middle is an implicit halting oracle. The LEM says that every logical statement is either true or false. This makes intuitive sense, but is wrong. If you think of logical statements as programs whose truth value we want to evaluate by executing a proof search, there are, in fact three “truth values”: True, false and uncomputable! This is a necessity because any axiom system worth its salt is Turing complete (this is basically what Gödel showed in his incompleteness theorems, he used Gödel numbers because Turing machines didn’t exist yet to formalize the same idea) and therefor has programs that don’t halt. Intuitionistic Logic (the logic we tend to formalize type theory and computer science with) doesn’t have this problem of an implicit halting oracle, and in my humble opinion should be used for the formalization of mathematics, on peril of trading infinite universes for an avocado sandwich and a big lizard if we use classical logic.
Note that us using constructivist/intuitionistic logic does not mean that “infinities aren’t a thing”, it’s a bit more subtle than that (and something I have admittedly not fully deconfused for myself yet). But basically, the kind of “infinities” that cosmologists talk about are (in my ontology) very different from the “super-infinities” that you get in the limit of hypercomputation. Intuitively, it’s important to differentiate “inductive infinities” (“you need arbitrarily many steps to complete this computation”) and “real infinities” (“the solution only pops out after infinity steps have been complete” i.e. a halting oracle).
The difference makes the most sense from the perspective of computational complexity theory. The universe is a “program” of complexity class PTIME/BQP (BQP is basically just the quantum version of PTIME), which means that you can evaluate the “next state” of the universe with at most PTIME/BQP computation. Importantly, this means that even if the universe is inflationary and “infinite”, you could evaluate the state of any part of it in (arbitrarily large) finite time. There are no “effects that emerge only at infinity”. The (evaluation of a given arbitrary state of the) universe halts. This is very different to the kinds of computations a hypercomputer is capable of (and less paradoxical). Which is why I found the following very amusing:
Quite the opposite! Or rather, one of those three things is not like the other. baby-universes are in P/BQP, wormholes are in PSPACE (assuming by wormholes you mean closed timelike curves, which is afaik the default interpretation), and hyper-computers are halting-complete which is ludicrously insanely not even remotely like the other two things. So in that regard, yes, I think consciousness being equal to cheesy-bread is more likely than finding a hypercomputer!
To be clear when I talk about “non-constructive logic is Bad™” I don’t mean that the actual literal symbolic mathematics is somehow malign (of course), it’s the interpretation we assign to it. We think we’re reasoning about infinite objects, but we’re really reasoning about computable weaker versions of the objects, and these are not the same thing. If one is maximally careful with ones interpretations, this is (theoretically) not a problem, but this is such a subtle difference of interpretation that this is very difficult to disentangle in our mere human minds. I think this is at the heart of the problems with infinite ethics, because understanding what the correct mathematical interpretations are is so damn subtle and confusing, we find ourselves in bizarre scenarios that seem contradictory and insane because we accidentally naively extrapolate interpretations to objects they don’t belong to.
I didn’t do the best of jobs formally arguing for my point, and I’m honestly still 20% confused about this all (at least), but I hope I at least gave some interesting intuitions about why the problem might be in our philosophy of mathematics, not our philosophy of ethics.
P.S. I’m sure you’ve heard of it before, but on the off chance you haven’t, I can not recommend this wonderful paper by Scott Aaronson highly enough for a crash course in many of these kinds of topics relevant to philosophers.
The case for observing a hypercomputer might rather be that a claim that has infinidesimal credence requires infinite amounts of proof to get to a finite credence level. So a being that can only entertain finite evidence would treat that credence as effectively zero but it might technically be separate from zero.
I could imagine programming a hypertask into an object and finding some exotic trajectory with proper time more than a finite amount and receive the object from such trajectory having completed the task. The hypothesis that it was actually a very potent classical computer is ruled out by the structure of the task. I am not convinced that the main or only method of checking for nature of computation is to check output bit by bit.
This seems dubious. Compare: “the actual credence that the universe contains more computing power than my brain is zero, because an observer with the computing power of my brain can never observe such an object in such a way that differentiates it from a brain-sized approximation”. It’s true that a bounded approximation to Solomonoff induction would think this way, but that seems like a problem with Solomonoff induction, not a guide for the way we should reason ourselves. See also the discussion here on forms of hypercomputation that could be falsified in principle.
Sorry, I previously assigned hypercomputers a non-zero credence, and you’re asking me to assign it zero credence. This requires an infinite amount of bits to update, which is impossible to collect in my computationally bounded state. Your case sounds sensible, but I literally can’t receive enough evidence over the course of a lifetime to be convinced by it.
Like, intuitively, it doesn’t feel literally impossible that humanity discovers a computationally unbounded process in our universe. If a convincing story is fed into my brain, with scientific consensus, personally verifying the math proof, concrete experiments indicating positive results, etc., I expect I would believe it. In my state of ignorance, I would not be surprised to find out there’s a calculation which requires a computationally unbounded process to calculate but a bounded process to verify.
To actually intuitively give something 0 (or 1) credence, though, to be so confident in a thesis that you literally can’t change your mind, that at the very least seems very weird. Self-referentially, I won’t actually assign that situation 0 credence, but even if I’m very confident that 0 credence is correct, my actual credence will be bounded by my uncertainty in my method of calculating credence.