When I think about the challenges with applying Solomonoff induction in practice, which the scientific method was designed around,
It’s hardly credible that the scientific method dating back about two hundred years , was consciously designed around Solomonof induction, published 1960ish, and dealing with computer programmes,which didn’t even exist until the mid twentieth century.
Ray Solomonoff’s definition doesn’t suggest that SI, even if you could overcome the computational limitations, is a general purpose truth finder.
“Solomonoff’s theory of inductive inference is a mathematical proof that if a universe is generated by an algorithm, then observations of that universe, encoded as a dataset, are best predicted by the smallest executable archive of that dataset”
So , to start with , it only works in computable universes.
on top of that, the relationship between a programme, a list of instructions, and a hypothesis as a description of reality, is not clear.
It’s easy to see how equivalence of programmes hypotheses holds , when hypothesis is used in the sense of. Bayesian hypothesis, that only predicts expect s future observations : the output of a candidate programme is the expected future observations. But the strong claims about SI require to do more than prediction, to tell you what reality is. But programmes are lists of commands ,such as “add one to register A” . They are imperative, not descriptive. So how do they make statements about the external world? Indeed, an SI guaranteed to be wrong about reality if you run it in a simulated world: it will find the programme that generates the simulation but won’t notice it is a simulation. This problem is hard to see for some people because of the ambiguity of terms.like “true”, “model” , “hypothesis” .. none of them make it clear whether what is being talked.about is predictive accuracy, or correspondence to reality.
I suppose you could be using Yudkowsky ’s version of Solomonoff as a metaphor for science—formulating. and testing hypotheses. Not a good metaphor, because it elides the importance of hypothesis generation—Science cannot find the truth without good hypotheses, and hypothesis formation is not a blind mechanistic process. As Richard rightly says.
“If every PhD in fundamental physics had contributed even one bit of usable evidence about how to unify quantum physics and general relativity, we’d have solved quantum gravity many times over by now. But we haven’t, because almost all of the work of science is in constructing sophisticated models, which Bayesianism says almost nothing about. (Formalisms like Solomonoff induction attempt to sidestep this omission by enumerating and simulating all computable models, but that’s so different from what any realistic agent can do that we should think of it less as idealized cognition and more as a different thing altogether, which just happens to converge to the same outcome in the infinite limit.)”
The second is that parsimony / Occam’s razor / Solomonoff prior is central to finding the truth, but scientists range from being imperfect at assessing the complexity-vs-parsimony of a theory, to being atrociously bad at it.
What are some examples of atrociously bad estimates of parsimony?
So the scientific enterprise is set up to rely as little as possible at complexity-assessments.
Is it? I can’t say I have noticed.
So anyway, if I were working on this project, the first thing I would try is to say that the “ideal” is Solomonoff induction searching for a true hypothesis
It’s all very well to talk about using SI to search for truth, but it’s not clear that it is capable of doing that, since all it is doing directly is rejecting programmes that don’t match observation,.and there is more to truth than matching observation.
Science students are always taught that that empirical testing is the hallmark of scientific truth, and are usually taught that Science delivers truths about reality—instrumentalism and anti realism being minority interests. But correspondence cannot be observed and is not tested directly. Naive scientism is naive because it has failed to notice the problem “Just look” is the first step in the scientific method, not the whole thing.
There must be some relationship between predictive accuracy and ultimate truth. Well, there is an obvious one ,and it’s the fact that a nonpredictive theory can’t be true. But it doesn’t have the corollary that a more predictively accurate theory is more correspondent. Ontologically wrong theories can be very accurate.
For instance, the Ptolemaic system can be made as accurate as you want for generating predictions, by adding extra epicycles … although it is false, in the sense of lacking ontological accuracy, since epicycles don’t exist. In fact, the more epicycles you add, the more accurate the model gets, and the less truthful to reality
Scientific theories minimally predict observations. Figuring out what the nature of the observed phenomenon is,is another matter. Induction can tell you the sun will rise in the east, but not that it is a fusion reactor. inference to the best explanation can tell you it is a fusion reactor, but leave fundamental ontological problems, like “what is a quark really”, unsolved. Reductionism.is a blessing and a curse—the curse is that when you reach the lowest level , you can no longer answer a “what is an X” question by specifying a bunch of components and their structure.
The problem of interpreting a fundamental theory is the problem of finding its ontological (or metaphysical) implications (including the option of treating some of its features as bookkeeping devices or otherwise in the map but the territory). We don’t live in the most convenient universe, the one where there is always a clinching difference in predictions. The persistence of the problem of interpreting quantum mechanics shows that.
@Tim H
Parsimony should be understood as merely a heuristic for how well a model could have predicted held out data
Why? Since predictive accuracy underdetermines metaphysical truth, there is a need from something additional to bridge them, then parsimony could well fit the bill
It’s hardly credible that the scientific method dating back about two hundred years , was consciously designed around Solomonof induction, published 1960ish, and dealing with computer programmes,which didn’t even exist until the mid twentieth century.
Ray Solomonoff’s definition doesn’t suggest that SI, even if you could overcome the computational limitations, is a general purpose truth finder.
“Solomonoff’s theory of inductive inference is a mathematical proof that if a universe is generated by an algorithm, then observations of that universe, encoded as a dataset, are best predicted by the smallest executable archive of that dataset”
So , to start with , it only works in computable universes.
on top of that, the relationship between a programme, a list of instructions, and a hypothesis as a description of reality, is not clear.
It’s easy to see how equivalence of programmes hypotheses holds , when hypothesis is used in the sense of. Bayesian hypothesis, that only predicts expect s future observations : the output of a candidate programme is the expected future observations. But the strong claims about SI require to do more than prediction, to tell you what reality is. But programmes are lists of commands ,such as “add one to register A” . They are imperative, not descriptive. So how do they make statements about the external world? Indeed, an SI guaranteed to be wrong about reality if you run it in a simulated world: it will find the programme that generates the simulation but won’t notice it is a simulation. This problem is hard to see for some people because of the ambiguity of terms.like “true”, “model” , “hypothesis” .. none of them make it clear whether what is being talked.about is predictive accuracy, or correspondence to reality.
I suppose you could be using Yudkowsky ’s version of Solomonoff as a metaphor for science—formulating. and testing hypotheses. Not a good metaphor, because it elides the importance of hypothesis generation—Science cannot find the truth without good hypotheses, and hypothesis formation is not a blind mechanistic process. As Richard rightly says.
“If every PhD in fundamental physics had contributed even one bit of usable evidence about how to unify quantum physics and general relativity, we’d have solved quantum gravity many times over by now. But we haven’t, because almost all of the work of science is in constructing sophisticated models, which Bayesianism says almost nothing about. (Formalisms like Solomonoff induction attempt to sidestep this omission by enumerating and simulating all computable models, but that’s so different from what any realistic agent can do that we should think of it less as idealized cognition and more as a different thing altogether, which just happens to converge to the same outcome in the infinite limit.)”
What are some examples of atrociously bad estimates of parsimony?
Is it? I can’t say I have noticed.
It’s all very well to talk about using SI to search for truth, but it’s not clear that it is capable of doing that, since all it is doing directly is rejecting programmes that don’t match observation,.and there is more to truth than matching observation.
Science students are always taught that that empirical testing is the hallmark of scientific truth, and are usually taught that Science delivers truths about reality—instrumentalism and anti realism being minority interests. But correspondence cannot be observed and is not tested directly. Naive scientism is naive because it has failed to notice the problem “Just look” is the first step in the scientific method, not the whole thing.
There must be some relationship between predictive accuracy and ultimate truth. Well, there is an obvious one ,and it’s the fact that a nonpredictive theory can’t be true. But it doesn’t have the corollary that a more predictively accurate theory is more correspondent. Ontologically wrong theories can be very accurate.
For instance, the Ptolemaic system can be made as accurate as you want for generating predictions, by adding extra epicycles … although it is false, in the sense of lacking ontological accuracy, since epicycles don’t exist. In fact, the more epicycles you add, the more accurate the model gets, and the less truthful to reality
Scientific theories minimally predict observations. Figuring out what the nature of the observed phenomenon is,is another matter. Induction can tell you the sun will rise in the east, but not that it is a fusion reactor. inference to the best explanation can tell you it is a fusion reactor, but leave fundamental ontological problems, like “what is a quark really”, unsolved. Reductionism.is a blessing and a curse—the curse is that when you reach the lowest level , you can no longer answer a “what is an X” question by specifying a bunch of components and their structure.
The problem of interpreting a fundamental theory is the problem of finding its ontological (or metaphysical) implications (including the option of treating some of its features as bookkeeping devices or otherwise in the map but the territory). We don’t live in the most convenient universe, the one where there is always a clinching difference in predictions. The persistence of the problem of interpreting quantum mechanics shows that.
@Tim H
Why? Since predictive accuracy underdetermines metaphysical truth, there is a need from something additional to bridge them, then parsimony could well fit the bill
We actually have, in the sense that we have discovered many forms of string theory and they are all consistent theories of quantum gravity.