I was musing on the old joke about anti-Occamian priors or anti-induction: ‘why are they sure it’s a good idea? Well, it’s never worked before.’ Obviously this is a bad idea for our kind of universe, but what kind of universe does it work in?
Well, in what sort of universe would every failure of X to appear that time interval make X that much more likely? It sounds a bit vaguely like the hope function but actually sounds more like an urn of balls where you sample without replace: every ball you pull (and discard) without finding X makes you a little more confident that next time will be X. Well, what kind of universe sees its possibilities shrinking every time?
For some reason, entropy came to mind. Our universe moves from low to high entropy, and we use induction. If a universe moved the opposite direction from high to low entropy, would its minds use anti-induction? (Minds seem like they’d be possible, if odd; our minds require local lowering of entropy to operate in an environment of increasing entropy, so why not anti-minds which require local raising of entropy to operate in an environment of decreasing entropy—somewhat analogous to reversible computers expending energy to erase bits.)
I have no idea if this makes any sense. (To go back to the urn model, I was thinking of it as sort of a cellular automaton mental model where every turn the plane shrinks: if you are predicting a glider as opposed to a huge turing machine, as every turn passes and the plane shrinks, the less you would expect to see the turing machine survive and the more you would expect to see a glider show up. Or if we were messing with geometry, it’d be as if we were given a heap of polygons with thousands of sides where every second a side was removed, and predicted a triangle—as the seconds pass, we don’t see any triangles, but Real Soon Now… Or to put it another way, as entropy decreases, necessarily fewer and fewer arrangements show up; particular patterns get jettisoned out as entropy shrinks, and so having observed a particular pattern, it’s unlikely to sneak back in: if the whole universe freezes into one giant simple pattern, the anti-inductionist mind would be quite right to have expected all but one observations to not repeat. Unlike our universe, where there seem to be ever more arrangements as things settle into thermal noise: if a arrangement shows up we’ll be seeing a lot of it around. Hence, we start with simple low entropy predictions and decreases confidence.)
Boxo suggested that anti-induction might be formalizable as the opposite of Solomonoff induction, but I couldn’t see how that’d work: if it simply picks the opposite of a maximizing AIXI and minimizes its score, then it’s the same thing but with an inverse utility function.
The other thing was putting a different probability distribution over programs, one that increases with length. But while you are forbidden uniform distributions over all the infinite integers, and you can have non-uniform decreasing distributions (like the speed prior or random exponentials), it’s not at all obvious what a non-uniform increasing distribution looks like—apparently it doesn’t work to say ‘infinite-length programs have p=0.5, then infinity-1 have p=0.25, then infinity-2 have p=0.125… then programs of length 1⁄0 have p=0’.
I was musing on the old joke about anti-Occamian priors or anti-induction: ‘why are they sure it’s a good idea? Well, it’s
never worked before.’ Obviously this is a bad idea for our kind of universe, but what kind of universe does it work in?
How can they possibly know/think that ‘it’ has never worked before? That assumes reliability of memory/data storage devices.
I don’t see how these anti-Occamians can ever conclude that data storage is reliable.
If they believe data storage is reliable, they can infer whether or not data storage worked in the past. If it worked, then data storage is probably not reliable now. If it didn’t work then it didn’t record correct information about the past. In neither case is the data storage reliable.
(An increasing probability distribution over the natural numbers is impossible. The sequence (P(1), P(2),...) would have to 1) be increasing 2) contain a nonzero element 3) sum to 1, which is impossible.)
I was musing on the old joke about anti-Occamian priors or anti-induction: ‘why are they sure it’s a good idea? Well, it’s never worked before.’ Obviously this is a bad idea for our kind of universe, but what kind of universe does it work in?
Well, in what sort of universe would every failure of X to appear that time interval make X that much more likely? It sounds a bit vaguely like the hope function but actually sounds more like an urn of balls where you sample without replace: every ball you pull (and discard) without finding X makes you a little more confident that next time will be X. Well, what kind of universe sees its possibilities shrinking every time?
For some reason, entropy came to mind. Our universe moves from low to high entropy, and we use induction. If a universe moved the opposite direction from high to low entropy, would its minds use anti-induction? (Minds seem like they’d be possible, if odd; our minds require local lowering of entropy to operate in an environment of increasing entropy, so why not anti-minds which require local raising of entropy to operate in an environment of decreasing entropy—somewhat analogous to reversible computers expending energy to erase bits.)
I have no idea if this makes any sense. (To go back to the urn model, I was thinking of it as sort of a cellular automaton mental model where every turn the plane shrinks: if you are predicting a glider as opposed to a huge turing machine, as every turn passes and the plane shrinks, the less you would expect to see the turing machine survive and the more you would expect to see a glider show up. Or if we were messing with geometry, it’d be as if we were given a heap of polygons with thousands of sides where every second a side was removed, and predicted a triangle—as the seconds pass, we don’t see any triangles, but Real Soon Now… Or to put it another way, as entropy decreases, necessarily fewer and fewer arrangements show up; particular patterns get jettisoned out as entropy shrinks, and so having observed a particular pattern, it’s unlikely to sneak back in: if the whole universe freezes into one giant simple pattern, the anti-inductionist mind would be quite right to have expected all but one observations to not repeat. Unlike our universe, where there seem to be ever more arrangements as things settle into thermal noise: if a arrangement shows up we’ll be seeing a lot of it around. Hence, we start with simple low entropy predictions and decreases confidence.)
Boxo suggested that anti-induction might be formalizable as the opposite of Solomonoff induction, but I couldn’t see how that’d work: if it simply picks the opposite of a maximizing AIXI and minimizes its score, then it’s the same thing but with an inverse utility function.
The other thing was putting a different probability distribution over programs, one that increases with length. But while you are forbidden uniform distributions over all the infinite integers, and you can have non-uniform decreasing distributions (like the speed prior or random exponentials), it’s not at all obvious what a non-uniform increasing distribution looks like—apparently it doesn’t work to say ‘infinite-length programs have p=0.5, then infinity-1 have p=0.25, then infinity-2 have p=0.125… then programs of length 1⁄0 have p=0’.
How can they possibly know/think that ‘it’ has never worked before? That assumes reliability of memory/data storage devices.
I don’t see how these anti-Occamians can ever conclude that data storage is reliable.
If they believe data storage is reliable, they can infer whether or not data storage worked in the past. If it worked, then data storage is probably not reliable now. If it didn’t work then it didn’t record correct information about the past. In neither case is the data storage reliable.
(An increasing probability distribution over the natural numbers is impossible. The sequence (P(1), P(2),...) would have to 1) be increasing 2) contain a nonzero element 3) sum to 1, which is impossible.)