How do you design the seed program for van Neumann probes that implement good infrastructure and decision theory to the end of physical and logical time in the reachable universe on cold computers or whatever we run as infrastructure. Seems like a difficult design problem to me. I don’t find it obvious whether the first entity that will attempt this won’t be burning the cosmic commons. So I would rather establish common knowledge in advance that there is a difficult problem to fix here and lots of shared gains to be had. I suspect most people care much less a about the cosmic commons than I do out of time myopia and genuine value differences.
How do you design the seed program for van Neumann probes that implement good infrastructure and decision theory to the end of physical and logical time in the reachable universe on cold computers or whatever we run as infrastructure. Seems like a difficult design problem to me. I don’t find it obvious whether the first entity that will attempt this won’t be burning the cosmic commons. So I would rather establish common knowledge in advance that there is a difficult problem to fix here and lots of shared gains to be had. I suspect most people care much less a about the cosmic commons than I do out of time myopia and genuine value differences.