One possible objection: what if humans are doing hypercomputation? E.g., being created by evolution (which is fundamentally “tied into” reality’s computation) lets humans tap into the latent computation of the universe in a way that an algorithmic AI can’t emulate, so it keeps humans around to use as hypercomputers. Various people have proposed similar hypotheses. I think this objection can be met, though.
The usual anti-Penrose point comes to mind: if quantum microtubules are really that useful, we can probably just build them into chips, and better, and the problem goes away.
Unless you mean the “tieing into” somehow requires a prefrontal cortex, at least 1 kidney, a working gallbladder, etc, in which case I think that’s just sheer privileging of hypothesis with not a scrap of evidence for it.
Former, not the latter. And yes, the anti-Penrose point applies, but we can skirt it by postulating that the superintelligence is limited in its decision theory—it can recognize good results when it seems them, much as TDT can recognize that UDT beats it at counterfactual mugging, but it’s architecturally constrained not to self-modify into the winning thing. So humans might run native hypercomputation or native super-awesome decision theory that an AI could exploit but that the AI would know it couldn’t emulate given its knowledge of its own limited architecture.
I guess you’re distantly alluding to the old discussion of ‘what would AIXI do if it ran into a hypercomputing oracle?’ in modern guise. I’m afraid I know too little about TDT or UDT to appreciate the point. It just seems a little far-fetched—so not only are we thinking about hypercomputation, which I believe is generally regarded as being orders of magnitude less likely than say P=NP, we’re also thinking about a superintelligent and superpowerful agent with a decision theory that just happens to be broken in the right way?
If we were being mined for our computational potential, I can’t help but feel human lives ought to be less repetitive than they are.
I believe is generally regarded as being orders of magnitude less likely than say P=NP
Haven’t seen any surveys, but I don’t think so. I think hypercomputation is considered by some important people to be more likely than P=NP. I believe very few people have really considered it, so you shouldn’t take anyone’s off-the-cuff impressions as meaning very much unless you know they’ve thought a lot about the limitations of theoretical computer science. I don’t really have any ax to grind on the matter, but I think hypercomputation is neglected.
we’re also thinking about a superintelligent and superpowerful agent with a decision theory that just happens to be broken in the right way?
I think my points were supposed to be disjunctive, not conjunctive. A broken decision theory or a limited theory of computation can both result in humans outcompeting superintelligences on certain very specific decision problems or (pseudo-)computations. Wei Dai’s “Metaphilosophical Mysteries” is relevant.
If we were being mined for our computational potential, I can’t help but feel human lives ought to be less repetitive than they are.
Given some models, yes. Given other models, the AI might not be able to locate what parts of the system have the special sauce and what parts don’t, so it’s more likely to let humans be.
Your link isn’t a stupid person, but to some extent, the lack of interest in hypercomputation says what the field thinks of it. Compare it to quantum computation, where people were avidly researching it and coming up with algorithms decades before even toy quantum computers showed up in cutting-edge labs.
Not sure, but it seems that whenever I get into discussions with you it’s usually about some potentially-important edge case or something. Strange.
But anyway, yeah. I just want to flag hypercomputation as a speculative thing that it might be worth taking an interest in, much like mirror matter. One or two of my default models are probably very similar to yours when it comes down to betting odds.
Compare it to quantum computation, where people were avidly researching it and coming up with algorithms decades before even toy quantum computers showed up in cutting-edge labs.
But only after it was discovered that the theory of quantum mechanics implied it was theoretically possible.
Compare it to quantum computation, where people were avidly researching it and coming up with algorithms decades before even toy quantum computers showed up in cutting-edge labs.
My understanding of the history is that everyone believed the extended Church-Turing thesis until someone noticed that the (already established) theory of quantum mechanics contradicted it.
I meant “apply” in the sense that one applies a mathematical model to a phenomenon. Specifically, it was implicitly assumed the the notion of polynomial time captured what was actually possible to compute in polynomial time.
It just seems a little far-fetched—so not only are we thinking about hypercomputation, which I believe is generally regarded as being orders of magnitude less likely than say P=NP
Um, you do realize you’re comparing apples and oranges there, since one is a statement about physics and the other a statement about mathematics.
As someone who understands computational theory, I strongly suspect you’re seriously confused about how computational complexity theory works. As I don’t have the time or interest to give a course in computational complexity, might I recommend asking the original question on mathoverflow if you are interested.
One possible objection: what if humans are doing hypercomputation? E.g., being created by evolution (which is fundamentally “tied into” reality’s computation) lets humans tap into the latent computation of the universe in a way that an algorithmic AI can’t emulate, so it keeps humans around to use as hypercomputers. Various people have proposed similar hypotheses. I think this objection can be met, though.
The usual anti-Penrose point comes to mind: if quantum microtubules are really that useful, we can probably just build them into chips, and better, and the problem goes away.
Unless you mean the “tieing into” somehow requires a prefrontal cortex, at least 1 kidney, a working gallbladder, etc, in which case I think that’s just sheer privileging of hypothesis with not a scrap of evidence for it.
Former, not the latter. And yes, the anti-Penrose point applies, but we can skirt it by postulating that the superintelligence is limited in its decision theory—it can recognize good results when it seems them, much as TDT can recognize that UDT beats it at counterfactual mugging, but it’s architecturally constrained not to self-modify into the winning thing. So humans might run native hypercomputation or native super-awesome decision theory that an AI could exploit but that the AI would know it couldn’t emulate given its knowledge of its own limited architecture.
I guess you’re distantly alluding to the old discussion of ‘what would AIXI do if it ran into a hypercomputing oracle?’ in modern guise. I’m afraid I know too little about TDT or UDT to appreciate the point. It just seems a little far-fetched—so not only are we thinking about hypercomputation, which I believe is generally regarded as being orders of magnitude less likely than say P=NP, we’re also thinking about a superintelligent and superpowerful agent with a decision theory that just happens to be broken in the right way?
If we were being mined for our computational potential, I can’t help but feel human lives ought to be less repetitive than they are.
Haven’t seen any surveys, but I don’t think so. I think hypercomputation is considered by some important people to be more likely than P=NP. I believe very few people have really considered it, so you shouldn’t take anyone’s off-the-cuff impressions as meaning very much unless you know they’ve thought a lot about the limitations of theoretical computer science. I don’t really have any ax to grind on the matter, but I think hypercomputation is neglected.
I think my points were supposed to be disjunctive, not conjunctive. A broken decision theory or a limited theory of computation can both result in humans outcompeting superintelligences on certain very specific decision problems or (pseudo-)computations. Wei Dai’s “Metaphilosophical Mysteries” is relevant.
Given some models, yes. Given other models, the AI might not be able to locate what parts of the system have the special sauce and what parts don’t, so it’s more likely to let humans be.
Your link isn’t a stupid person, but to some extent, the lack of interest in hypercomputation says what the field thinks of it. Compare it to quantum computation, where people were avidly researching it and coming up with algorithms decades before even toy quantum computers showed up in cutting-edge labs.
Wei Dai’s link is pretty controversial.
Not sure, but it seems that whenever I get into discussions with you it’s usually about some potentially-important edge case or something. Strange.
But anyway, yeah. I just want to flag hypercomputation as a speculative thing that it might be worth taking an interest in, much like mirror matter. One or two of my default models are probably very similar to yours when it comes down to betting odds.
But only after it was discovered that the theory of quantum mechanics implied it was theoretically possible.
My understanding of the history is that everyone believed the extended Church-Turing thesis until someone noticed that the (already established) theory of quantum mechanics contradicted it.
I don’t think I’ve ever seen anyone invoke the extended Church-Turing thesis by either name or substance before quantum computing came around.
People were talking about P-time before quantum computing and implicitly assuming that it applied to any computer they could build.
I don’t see how one would apply “P-time” to “any computer they could build”.
I meant “apply” in the sense that one applies a mathematical model to a phenomenon. Specifically, it was implicitly assumed the the notion of polynomial time captured what was actually possible to compute in polynomial time.
Um, you do realize you’re comparing apples and oranges there, since one is a statement about physics and the other a statement about mathematics.
In this area, I do not think there is such a hard and fast distinction.
So, how would you phrase the existence of hypercomputation as a mathematical statement?
Presumably something involving recursively enumerable functions...
As someone who understands computational theory, I strongly suspect you’re seriously confused about how computational complexity theory works. As I don’t have the time or interest to give a course in computational complexity, might I recommend asking the original question on mathoverflow if you are interested.
Apologies if that came off as rude.