Incidentally, I’m also interested in what specifically you mean by “random program”. A natural interpretation is that you’re talking about a program that is drawn from some kind of distribution across the set of all possible programs, but as far as I can tell, you haven’t actually defined said distribution. Without a specific distribution to talk about, any claim about how likely a “random program” is to do anything is largely meaningless, since for any such claim, you can construct a distribution that makes that claim true.
(Note: The above paragraph was originally a parenthetical note on my other reply, but I decided to expand it into its own, separate comment, since in my experience having multiple unrelated discussions in a single comment chain often leads to unproductive conversation.)
Well, good question. Frankly I don’t think it matters. I don’t believe that my claims are sensitive to the distributions (aside from some convoluted ones), or that giving you a specific distribution would help you to defend either position (feel free to prove me wrong). But when I want to feel rigorous, I assume that I’m starting off with a natural length-based distribution over all Turing machines (or maybe all neural networks), then discard all machines that fail to pass some relatively simple criteria about the output they generate (e.g. does it classify a given set of cat pictures correctly), keep the ones that passed, normalize and draw from that.
But really, by “random” I mean nearly anything that’s not entirely intentional. To use a metaphor for machine learning, if you pick a random point in the world map, then find the nearest point that’s 2km above sea level, you’ll find a “random” point that’s 2km above sea level. The algorithm has a non-random step, but the outcome is clearly random in a significant way. The distribution you get is different from the one I described in my previous paragraph (where you just filtered the initial point distribution to get the points at 2km), but they’ll most likely be close.
Incidentally, I’m also interested in what specifically you mean by “random program”. A natural interpretation is that you’re talking about a program that is drawn from some kind of distribution across the set of all possible programs, but as far as I can tell, you haven’t actually defined said distribution. Without a specific distribution to talk about, any claim about how likely a “random program” is to do anything is largely meaningless, since for any such claim, you can construct a distribution that makes that claim true.
(Note: The above paragraph was originally a parenthetical note on my other reply, but I decided to expand it into its own, separate comment, since in my experience having multiple unrelated discussions in a single comment chain often leads to unproductive conversation.)
Well, good question. Frankly I don’t think it matters. I don’t believe that my claims are sensitive to the distributions (aside from some convoluted ones), or that giving you a specific distribution would help you to defend either position (feel free to prove me wrong). But when I want to feel rigorous, I assume that I’m starting off with a natural length-based distribution over all Turing machines (or maybe all neural networks), then discard all machines that fail to pass some relatively simple criteria about the output they generate (e.g. does it classify a given set of cat pictures correctly), keep the ones that passed, normalize and draw from that.
But really, by “random” I mean nearly anything that’s not entirely intentional. To use a metaphor for machine learning, if you pick a random point in the world map, then find the nearest point that’s 2km above sea level, you’ll find a “random” point that’s 2km above sea level. The algorithm has a non-random step, but the outcome is clearly random in a significant way. The distribution you get is different from the one I described in my previous paragraph (where you just filtered the initial point distribution to get the points at 2km), but they’ll most likely be close.
Maybe that answers your other comment too?