I love the idea behind this post, and I’ve come back to it many times over the years. Now I’m coming back to it with error correction on my mind.
I agree that in a game of telephone only information which is preserved through each step will make it to the end. But many processes are unlike this. E.g., normally if you didn’t hear something I said, you can ask. And one could imagine arbitrarily long gaps—you ask me to clarify in a week or a month or what have you. Or maybe someone has an insight and then loses it, gains it again, and then writes a book such that the information is pretty reliably transmitted in the world.
In all such cases there will be a few to many layers of the nested Markov structure which lose the information but gain it again at a later point. Which is to say that the constraint of needing each intermediate state to carry the information in order for the information to be preserved seems too strong. “Ability to recover the information” seems like a necessary constraint, although not sufficient: you need some kind of error corrector or the original generative process to run again in order to in fact recover it.
I love the idea behind this post, and I’ve come back to it many times over the years. Now I’m coming back to it with error correction on my mind.
I agree that in a game of telephone only information which is preserved through each step will make it to the end. But many processes are unlike this. E.g., normally if you didn’t hear something I said, you can ask. And one could imagine arbitrarily long gaps—you ask me to clarify in a week or a month or what have you. Or maybe someone has an insight and then loses it, gains it again, and then writes a book such that the information is pretty reliably transmitted in the world.
In all such cases there will be a few to many layers of the nested Markov structure which lose the information but gain it again at a later point. Which is to say that the constraint of needing each intermediate state to carry the information in order for the information to be preserved seems too strong. “Ability to recover the information” seems like a necessary constraint, although not sufficient: you need some kind of error corrector or the original generative process to run again in order to in fact recover it.