Ha, I was just about to write this post.
To add something, I think you can justify the uniform measure on bounded intervals of reals (for illustration purposes, say [0,1]) by the following argument: “Measuring a real number x∈[0,1]” is obviously simply impossible if interpreted literally, containing an infinite amount of data. Instead this is supposed to be some sort of idealization of a situation where you can observe “as many bits as you want” of the binary expansion of the number (choosing another base gives the same measure). If you now apply the principle of indifference to each measured bit, you’re left with Lebesgue measure.
It’s not clear that there’s a “right” way to apply this type of thinking to produce “the correct” prior on N (or R or any other non-compact space.
Given any particular admissible representation of a topological space, I do agree you can generate a Borel probability measure by pushing forward the Haar measure of the digit-string space ΣN (considered as a countable product of ω copies of Σ, considered as a group with the modular-arithmetic structure of Z/|Σ|) along the representation. This construction is studied in detail in (Mislove, 2015).
But, actually, the representation itself (in this case, the Cantor map) smuggles in Lebesgue measure, because each digit happens to cut the target space “in half” according to Lebesgue measure. If I postcompose, say, x↦√x after the Cantor map, that is also an admissible representation of [0,1], but it no longer induces Lebesgue measure. This works for any continuous bijection, so any absolutely continuous probability measure on [0,1] can be induced by such a representation. In fact, this is why the inverse-CDF algorithm for drawing samples from arbitrary distributions, given only uniform random bits, works.
That being said, you can apply this to non-compact spaces. I could get a probability measure on R via a decimal representation, where, say, the number of leading zeros encodes the exponent in unary and the rest is the mantissa. [Edit: I didn’t think this through all the way, and it can only represent real numbers ≥1. No big obstacle; post-compose x↦log(x−1).] The reason there doesn’t seem to be a “correct” way to do so is that, because there’s no Haar probability measure on non-compact spaces (at least, usually?), there’s no digit representation that happens to cut up the space “evenly” according to such a canonical probability measure.
Right, that’s essentially what I mean. You’re of course right that this doesn’t let you get around the existence of nonmeasurepreserving automorphisms. I guess what I’m saying is that, if you’re trying to find a prior on [0,1], you should try to think about what system of finite measurements this is idealizing, and see if you can apply a symmetry argument to those bits. Which isn’t always the case! You can only apply the principle of indifference if you’re actually indifferent. But if it’s the case that X∈[0,1] is generated in a way where “there’s no reason to suspect that any bit should be 0 rather than 1, or that there should be correlations between the bits”, then it’s of course not the case that √X has this same property. But of course you still need to look at the actual situation to see what symmetry exists or doesn’t exist.
Haar measures are a high-powered way of doing this, I was just thinking about taking the iterated product measure of the uniform probability measure on {0,1} (justified by symmetry considerations). You can of course find maps from {0,1}∞ to all sorts of spaces but it seems harder to transport symmetry considerations along these maps.
Ha, I was just about to write this post. To add something, I think you can justify the uniform measure on bounded intervals of reals (for illustration purposes, say [0,1]) by the following argument: “Measuring a real number x∈[0,1]” is obviously simply impossible if interpreted literally, containing an infinite amount of data. Instead this is supposed to be some sort of idealization of a situation where you can observe “as many bits as you want” of the binary expansion of the number (choosing another base gives the same measure). If you now apply the principle of indifference to each measured bit, you’re left with Lebesgue measure.
It’s not clear that there’s a “right” way to apply this type of thinking to produce “the correct” prior on N (or R or any other non-compact space.
Given any particular admissible representation of a topological space, I do agree you can generate a Borel probability measure by pushing forward the Haar measure of the digit-string space ΣN (considered as a countable product of ω copies of Σ, considered as a group with the modular-arithmetic structure of Z/|Σ|) along the representation. This construction is studied in detail in (Mislove, 2015).
But, actually, the representation itself (in this case, the Cantor map) smuggles in Lebesgue measure, because each digit happens to cut the target space “in half” according to Lebesgue measure. If I postcompose, say, x↦√x after the Cantor map, that is also an admissible representation of [0,1], but it no longer induces Lebesgue measure. This works for any continuous bijection, so any absolutely continuous probability measure on [0,1] can be induced by such a representation. In fact, this is why the inverse-CDF algorithm for drawing samples from arbitrary distributions, given only uniform random bits, works.
That being said, you can apply this to non-compact spaces. I could get a probability measure on R via a decimal representation, where, say, the number of leading zeros encodes the exponent in unary and the rest is the mantissa. [Edit: I didn’t think this through all the way, and it can only represent real numbers ≥1. No big obstacle; post-compose x↦log(x−1).] The reason there doesn’t seem to be a “correct” way to do so is that, because there’s no Haar probability measure on non-compact spaces (at least, usually?), there’s no digit representation that happens to cut up the space “evenly” according to such a canonical probability measure.
Right, that’s essentially what I mean. You’re of course right that this doesn’t let you get around the existence of nonmeasurepreserving automorphisms. I guess what I’m saying is that, if you’re trying to find a prior on [0,1], you should try to think about what system of finite measurements this is idealizing, and see if you can apply a symmetry argument to those bits. Which isn’t always the case! You can only apply the principle of indifference if you’re actually indifferent. But if it’s the case that X∈[0,1] is generated in a way where “there’s no reason to suspect that any bit should be 0 rather than 1, or that there should be correlations between the bits”, then it’s of course not the case that √X has this same property. But of course you still need to look at the actual situation to see what symmetry exists or doesn’t exist.
Haar measures are a high-powered way of doing this, I was just thinking about taking the iterated product measure of the uniform probability measure on {0,1} (justified by symmetry considerations). You can of course find maps from {0,1}∞ to all sorts of spaces but it seems harder to transport symmetry considerations along these maps.