The issues are whether the quantity, which you have called a probability , actually is probability , and whether the thing you at treating as a model of reality, is actually a such a model , in the sense of scientific realism, or merely something that churns out predictions, in the sense of instrumentalism.
I’m not quite sure how to respond to this; like, I think you’re right that SI is not solving the hard problem, but I think you’re wrong that SI is not solving the easy problem.
What are the hard and easy problems? Realism and instrumentalism? I haven’t said that SI is incapable of instrumentalism (prediction). Indeed, that might be the only thing it can do.
For example, I think the quantity actually is a probability, in that it satisfies all of the desiderata that probability theory places on quantities.
I think the mathematical constraints are clearly insufficient to show that something is a probability, even if they are necessary. If I have a cake of 1m^2, and I cut it up. Then the pieces sum to 1. But pieces of cake aren’t probabilities.
Do I think it’s the probability that the actual source code of the universe is that particular implementation? Well, it sure seems shaky to have as an axiom that God prefers shorter variable names, but since probability is in the mind, I don’t want to rule out any programs a priori, and there are more programs with longer variable names than programs with shorter variable names, I don’t see any other way to express what my state of ignorance would be given infinite cognitive resources.
So every hypothesis has the same probability of “not impossible”. Well, no, several times over. You haven’t shown that programmes are hypotheses, and what an SI is doing is assigning different non zero order probabilities, not a uniform one, and it is doing so based on programme length, although we don’t know that reality is a programme, and so on.
Also, I’m not really sure what a model of reality in the sense of scientific realism is,
Do you think scientists are equally troubled?
But are you saying that the “deep structure” is the ontologcal content?
My current suspicion is that we’re having this discussion, actually; it feels to me like if you were the hypercomputer running SI, you wouldn’t see the point of the ontological content; you could just integrate across all the hypotheses and have perfectly expressed your uncertainty about the world.
Even if I no longer have a instrumental need for something, I can terminally value it.
But it isn’t about me.
The rational sphere in general value realism, and make realistic claims. Yudkowsky has made claims about God not existing, and MWI being true that are explicitly based on SI style reasoning. So the cat is out of the bag… SI cannot be defended as something that was only ever intended as an instrumentalist predictor without walking back those claims.
But if you’re running an algorithm that uses caching or otherwise clumps things together, those intermediate variables feel like they’re necessary objects that need to be explained and generated somehow.
You’.re saying realism is an illusion? Maybe that’s your philosophy, but it’s not the less wrong philosophy.
[Like, it might be interesting to look at the list of outputs that a model in the sense of scientific realism could give you, and ask if SI could also give you those outputs with minimal adjustment.]
You haven’t shown that programmes are hypotheses, and what an SI is doing is assigning different non zero order probabilities, not a uniform one, and it is doing so based on programme length, although we don’t know that reality is a programme, and so on.
SI only works for computable universes; otherwise you’re out of luck. If you’re in an uncomputable universe… I’m not sure what your options are, actually. [If you are in a computable universe, then there must be a program that corresponds to it, because otherwise it would be uncomputable!]
You can’t assign a uniform probability to all the programs, because there are infinitely many, and while there is a mathematically well-defined “infinitely tall function” there isn’t a mathematically well-defined “infinitely flat function.”
[Like, I keep getting the sense you want SI to justify the assumption that God prefers shorter variable names, and not liking the boring answer that in a condition of ignorance, our guess has to be that way because there are more ways to have long variable names than short variable names, and that our guess isn’t really related to the implementation details of the actual program God wrote.]
Do you think scientists are equally troubled?
I mean, I don’t know how muscles work, outside of the barest details, and yet I can use mine just fine. If I had to build some without being able to order them from a catalog, I’d be in trouble. I think AI researchers trying to make artificial scientists are equally troubled, and that’s the standard I’m trying to hold myself to.
You’.re saying realism is an illusion?
I don’t know what you mean by realism or illusion? Like, there’s a way in which “my hand” is an illusion, in that it’s a concept that exists in my mind, only somewhat related to physical reality. When I probe the edges of the concept, I can see the seams and how it fails to correspond to the underlying reality.
Like, the slogan that seems relevant here is “the map is not the territory.” If realism means “there’s a territory out there”, I’m sold on there being a territory out there. If realism means “there is a map in the territory”, I agree in the boring sense that a map exists in your head which exists in the territory, but think that makes some classes of arguments about the map confused.
For example, I could try to get worked up over whether, when I put hand sanitizer on my hands, the sanitizer becomes “part of my hand”, but it seems wiser to swap out the coarse “hand” model with a finer “molecules and permeable tissue” model, where the “part of my hand” question no longer feels sensible, and “where are these molecules?” question does feel sensible.
One of the ways in which SI seems conceptually useful is that it lets me ask questions like “would different programs say ‘yes’ and ‘no’ to the question of ‘is the sanitizer part of my hand’?”. If I can’t ground out the question in some deep territorial way, then it feels like the question isn’t really about the territory.
SI only works for computable universes; otherwise you’re out of luck
SI cannot generate realistic hypotheses about uncomputable universes , but it doesn’t follow that it can generate realistic hypotheses about computable universes.
You can’t assign a uniform probability to all the programs, because there are infinitely many, and while there is a mathematically well-defined “infinitely tall function” there isn’t a mathematically well-defined “infinitely flat function.”
The fact that an SI must sort and filter candidate functions does not mean it s doing so according to probability.
our guess has to be that way because there are more ways to have long variables
Given the assumptions that you have an infinite number of prgrammes, and that you need to come to a determinate result in finite time, then you need to favour shorter programmes. That’s a reasonable justification for the operation of an SI which happens to have nothing to do truth or probability or reference or realism. (You lapsed into describing the quantity an SI sorts programmes by as “probability”...that has not, of course, been established)
If I can’t ground out the question in some deep territorial way, then it feels like the question isn’t really about the territory.
You haven’t shown that an SI is capable of anything deep and territorial. After all,it’s only trying to predict observations.
Given the assumptions that you have an infinite number of prgrammes, and that you need to come to a determinate result in finite time, then you need to favour shorter programmes.
If you need to come to a determinate result in a finite number of computational steps (my replacement for ‘time’), then SI isn’t the tool for you. It’s the most general and data-efficient predictor possible, at the cost of totally exploding the computational budget.
I think if you are trying to evaluate a finite set of programs in finite time, it’s not obvious that program length is the thing to sort them by; I think the speed prior makes more sense, and I think actual humans are doing something meaningfully different.
---
I currently don’t see all that much value in responding to “You haven’t shown / established” claims; like, SI is what it is, you seem to have strong opinions about how it should label particular things, and I don’t think those opinions are about the part of SI that’s interesting, or about why it’s only useful as a hypothetical model (I think attacks from this angle are more compelling on that front). If you’re getting value out of this exchange, I can give responding to your comments another go, but I’m not sure I have new things to say about the association between observations and underlying reality or aggregation of possibilities through the use of probabilities. (Maybe I have elaborations that would either more clearly convey my point, or expose the mistakes I’m making?)
If you need to come to a determinate result in a finite number of computational steps (my replacement for ‘time’), then SI isn’t the tool for you
It isn’t any a tool for anybody because it’s uncomputable. Whatever interest it has must be theoretical.
I’m responding to claims that SI can solve long standing philosophical puzzles such as the existence of God or the correct interpretation of quantum mechanics. The claims have been made, and they have been made here but they may not have been made by you.
I’m responding to claims that SI can solve long standing philosophical puzzles such as the existence of God or the correct interpretation of quantum mechanics.
Ah, I see. I’m not sure I would describe SI as ‘solving’ those puzzles, rather than recasting them in a clearer light.
Like, a program which contains Zeus and Hera will give rather different observations than a program which doesn’t. On the other hand, when we look at programs that give the same observations, one of which also simulates a causally disconnected God and the other of which doesn’t, then it should be clear that those programs look the same from our stream of observations (by definition!) and so we can’t learn anything about them through empirical investigation (like with p-zombies).
So in my mind, the interesting “theism vs. atheism?” question is the question of whether there are activist gods out there; if Ares actually exists, then you (probably) profit by not displeasing him. Beliefs should pay rent in anticipated experiences, which feels like a very SI position to have.
Of course, it’s possible to have a causally disconnected afterlife downstream of us, where things that we do now can affect it and nothing we do there can affect us now. [This relationship should already be familiar from the relationship between the past and the present.] SI doesn’t rule that out—it can’t until you get relevant observations!--but the underlying intuition notes that the causal disconnection makes it pretty hard to figure out which afterlife. [This is the response to Pascal’s Wager where you say “well, but what about anti-God, who sends you to hell for being a Christian and sends you to heaven for not being one?”, and then you get into how likely it is that you have an activist God that then steps back, and arguments between Christians as to whether or not miracles happen in the present day.]
But I think the actual productive path, once you’re moderately confident Zeus isn’t on Olympus, is not trying to figure out if invisi-Zeus is in causally-disconnected-Olympus, but looking at humans to figure out why they would have thought Zeus was intuitively likely in the first place; this is the dissolving the question approach.
With regard to QM, when I read through this post, it is relying pretty heavily on Occam’s Razor, which (for Eliezer at least) I assume is backed by SI. But it’s in the normal way where, if you want to postulate something other than the simplest hypothesis, you have to make additional choices, and that each choice that could have been different loses you points in the game of Follow-The-Improbability. But a thing that I hadn’t noticed before this conversation, which seems pretty interesting to me, is that whether you prefer MWI might depend on whether you use the simplicity prior or the speed prior, and then I think the real argument for MWI rests more on the arguments here than on Occam’s Razor grounds (except for the way in which you think a physics that follows all the same principles is more likely because of Occam’s Razor on principles, which might be people’s justification for that?).
Ah, I see. I’m not sure I would describe SI as ‘solving’ those puzzles, rather than recasting them in a clearer light
The claim has been made , even if you don’t believe it.
Beliefs should pay rent in anticipated experiences, which feels like a very SI position to have
Rationalists don’t consistently believe that, because if they did , they would be indfferent about MW versus Copenhagen , since all interpretations make the same predictions. Lesswrongian epistemology isn’t even consistent.
But I think the actual productive path, once you’re moderately confident Zeus isn’t on Olympus, is not trying to figure out if invisi-Zeus is in causally-disconnected-Olympus, but looking at humans to figure out why they would have thought Zeus was intuitively likely in the first place; this is the dissolving the question approach
If you can have a non empirical reason to believe in non interacting branches of the universal wave function, your theist opponents can have a non empirical reason to believe in non interacting gods.
With regard to QM, when I read through this post, it is relying pretty heavily on Occam’s Razor, which (for Eliezer at least) I assume is backed by SI
Of course not. SI can’t tell you why simplicity matters, epistemologically. At the same time. It is clear that simplicity is no additional help in making predictions. Once you have filtered out the non predictive order programmes, the remaining ones are all equally predictive … so whatever simpliciy is supplying, it isn’t extra productiveness. The obvious answer is that it’s some ability to show that, out of N equally predictive theories, one corresponds to reality.
That’s a standard defence of Occam’s razor. It isn’t given by SI, as we have seen. SI just needs the simplicity criterion in order to be able to spit something out.
But there are other defenses of Occam’s razor.
And the traditional versions don’t settle everything in favour of MWI and against (sophisticated versions of) God..those are open questions.
And SI isnt a new improved version of Occam’s razor. In fact , it is unable to relate simplicity to truth.
But a thing that I hadn’t noticed before this conversation, which seems pretty interesting to me, is that whether you prefer MWI might depend on whether you use the simplicity prior or the speed prior, a
These old problems are open problems because we can’t agree on which kind of simplicity is relevant. SI doesn’t help because it introduces yet another simplicity measure. Or maybe two,the speed prior and the space prior.
I think the real argument for MWI rests more on the arguments here
Wrongly conflates Copenhagen with Objective Reduction.
Wrongly assumes MW is the only alternative to “Copenhagen”.
What are the hard and easy problems? Realism and instrumentalism? I haven’t said that SI is incapable of instrumentalism (prediction). Indeed, that might be the only thing it can do.
I think the mathematical constraints are clearly insufficient to show that something is a probability, even if they are necessary. If I have a cake of 1m^2, and I cut it up. Then the pieces sum to 1. But pieces of cake aren’t probabilities.
So every hypothesis has the same probability of “not impossible”. Well, no, several times over. You haven’t shown that programmes are hypotheses, and what an SI is doing is assigning different non zero order probabilities, not a uniform one, and it is doing so based on programme length, although we don’t know that reality is a programme, and so on.
Do you think scientists are equally troubled?
Even if I no longer have a instrumental need for something, I can terminally value it.
But it isn’t about me.
The rational sphere in general value realism, and make realistic claims. Yudkowsky has made claims about God not existing, and MWI being true that are explicitly based on SI style reasoning. So the cat is out of the bag… SI cannot be defended as something that was only ever intended as an instrumentalist predictor without walking back those claims.
You’.re saying realism is an illusion? Maybe that’s your philosophy, but it’s not the less wrong philosophy.
It’s obvious that it could, but so what?
SI only works for computable universes; otherwise you’re out of luck. If you’re in an uncomputable universe… I’m not sure what your options are, actually. [If you are in a computable universe, then there must be a program that corresponds to it, because otherwise it would be uncomputable!]
You can’t assign a uniform probability to all the programs, because there are infinitely many, and while there is a mathematically well-defined “infinitely tall function” there isn’t a mathematically well-defined “infinitely flat function.”
[Like, I keep getting the sense you want SI to justify the assumption that God prefers shorter variable names, and not liking the boring answer that in a condition of ignorance, our guess has to be that way because there are more ways to have long variable names than short variable names, and that our guess isn’t really related to the implementation details of the actual program God wrote.]
I mean, I don’t know how muscles work, outside of the barest details, and yet I can use mine just fine. If I had to build some without being able to order them from a catalog, I’d be in trouble. I think AI researchers trying to make artificial scientists are equally troubled, and that’s the standard I’m trying to hold myself to.
I don’t know what you mean by realism or illusion? Like, there’s a way in which “my hand” is an illusion, in that it’s a concept that exists in my mind, only somewhat related to physical reality. When I probe the edges of the concept, I can see the seams and how it fails to correspond to the underlying reality.
Like, the slogan that seems relevant here is “the map is not the territory.” If realism means “there’s a territory out there”, I’m sold on there being a territory out there. If realism means “there is a map in the territory”, I agree in the boring sense that a map exists in your head which exists in the territory, but think that makes some classes of arguments about the map confused.
For example, I could try to get worked up over whether, when I put hand sanitizer on my hands, the sanitizer becomes “part of my hand”, but it seems wiser to swap out the coarse “hand” model with a finer “molecules and permeable tissue” model, where the “part of my hand” question no longer feels sensible, and “where are these molecules?” question does feel sensible.
One of the ways in which SI seems conceptually useful is that it lets me ask questions like “would different programs say ‘yes’ and ‘no’ to the question of ‘is the sanitizer part of my hand’?”. If I can’t ground out the question in some deep territorial way, then it feels like the question isn’t really about the territory.
SI cannot generate realistic hypotheses about uncomputable universes , but it doesn’t follow that it can generate realistic hypotheses about computable universes.
The fact that an SI must sort and filter candidate functions does not mean it s doing so according to probability.
Given the assumptions that you have an infinite number of prgrammes, and that you need to come to a determinate result in finite time, then you need to favour shorter programmes. That’s a reasonable justification for the operation of an SI which happens to have nothing to do truth or probability or reference or realism. (You lapsed into describing the quantity an SI sorts programmes by as “probability”...that has not, of course, been established)
You haven’t shown that an SI is capable of anything deep and territorial. After all,it’s only trying to predict observations.
If you need to come to a determinate result in a finite number of computational steps (my replacement for ‘time’), then SI isn’t the tool for you. It’s the most general and data-efficient predictor possible, at the cost of totally exploding the computational budget.
I think if you are trying to evaluate a finite set of programs in finite time, it’s not obvious that program length is the thing to sort them by; I think the speed prior makes more sense, and I think actual humans are doing something meaningfully different.
---
I currently don’t see all that much value in responding to “You haven’t shown / established” claims; like, SI is what it is, you seem to have strong opinions about how it should label particular things, and I don’t think those opinions are about the part of SI that’s interesting, or about why it’s only useful as a hypothetical model (I think attacks from this angle are more compelling on that front). If you’re getting value out of this exchange, I can give responding to your comments another go, but I’m not sure I have new things to say about the association between observations and underlying reality or aggregation of possibilities through the use of probabilities. (Maybe I have elaborations that would either more clearly convey my point, or expose the mistakes I’m making?)
It isn’t any a tool for anybody because it’s uncomputable. Whatever interest it has must be theoretical.
I’m responding to claims that SI can solve long standing philosophical puzzles such as the existence of God or the correct interpretation of quantum mechanics. The claims have been made, and they have been made here but they may not have been made by you.
Ah, I see. I’m not sure I would describe SI as ‘solving’ those puzzles, rather than recasting them in a clearer light.
Like, a program which contains Zeus and Hera will give rather different observations than a program which doesn’t. On the other hand, when we look at programs that give the same observations, one of which also simulates a causally disconnected God and the other of which doesn’t, then it should be clear that those programs look the same from our stream of observations (by definition!) and so we can’t learn anything about them through empirical investigation (like with p-zombies).
So in my mind, the interesting “theism vs. atheism?” question is the question of whether there are activist gods out there; if Ares actually exists, then you (probably) profit by not displeasing him. Beliefs should pay rent in anticipated experiences, which feels like a very SI position to have.
Of course, it’s possible to have a causally disconnected afterlife downstream of us, where things that we do now can affect it and nothing we do there can affect us now. [This relationship should already be familiar from the relationship between the past and the present.] SI doesn’t rule that out—it can’t until you get relevant observations!--but the underlying intuition notes that the causal disconnection makes it pretty hard to figure out which afterlife. [This is the response to Pascal’s Wager where you say “well, but what about anti-God, who sends you to hell for being a Christian and sends you to heaven for not being one?”, and then you get into how likely it is that you have an activist God that then steps back, and arguments between Christians as to whether or not miracles happen in the present day.]
But I think the actual productive path, once you’re moderately confident Zeus isn’t on Olympus, is not trying to figure out if invisi-Zeus is in causally-disconnected-Olympus, but looking at humans to figure out why they would have thought Zeus was intuitively likely in the first place; this is the dissolving the question approach.
With regard to QM, when I read through this post, it is relying pretty heavily on Occam’s Razor, which (for Eliezer at least) I assume is backed by SI. But it’s in the normal way where, if you want to postulate something other than the simplest hypothesis, you have to make additional choices, and that each choice that could have been different loses you points in the game of Follow-The-Improbability. But a thing that I hadn’t noticed before this conversation, which seems pretty interesting to me, is that whether you prefer MWI might depend on whether you use the simplicity prior or the speed prior, and then I think the real argument for MWI rests more on the arguments here than on Occam’s Razor grounds (except for the way in which you think a physics that follows all the same principles is more likely because of Occam’s Razor on principles, which might be people’s justification for that?).
The claim has been made , even if you don’t believe it.
Rationalists don’t consistently believe that, because if they did , they would be indfferent about MW versus Copenhagen , since all interpretations make the same predictions. Lesswrongian epistemology isn’t even consistent.
If you can have a non empirical reason to believe in non interacting branches of the universal wave function, your theist opponents can have a non empirical reason to believe in non interacting gods.
Of course not. SI can’t tell you why simplicity matters, epistemologically. At the same time. It is clear that simplicity is no additional help in making predictions. Once you have filtered out the non predictive order programmes, the remaining ones are all equally predictive … so whatever simpliciy is supplying, it isn’t extra productiveness. The obvious answer is that it’s some ability to show that, out of N equally predictive theories, one corresponds to reality.
That’s a standard defence of Occam’s razor. It isn’t given by SI, as we have seen. SI just needs the simplicity criterion in order to be able to spit something out.
But there are other defenses of Occam’s razor.
And the traditional versions don’t settle everything in favour of MWI and against (sophisticated versions of) God..those are open questions.
And SI isnt a new improved version of Occam’s razor. In fact , it is unable to relate simplicity to truth.
These old problems are open problems because we can’t agree on which kind of simplicity is relevant. SI doesn’t help because it introduces yet another simplicity measure. Or maybe two,the speed prior and the space prior.
Wrongly conflates Copenhagen with Objective Reduction.
Wrongly assumes MW is the only alternative to “Copenhagen”.