Thanks. I would add that my first point—that we should stay humble about what a superintelligence ought to do—also extends to how it ought to do it.
Suppose superintelligences do converge on the most extreme solution : a computronium bubble expanding at light speed. That still doesn’t imply they would convert the entire content of their light cone into a uniform computational medium visible from parsecs away. Speaking as a non-physicist, my impression is that the structures we see in the universe are not arbitrary, they exist because they are stable equilibria. Matter at cosmological scales seems to end up in a fairly short list: planets and chemically bound bodies held together by electromagnetic forces, stars supported by nuclear fusion, neutron stars supported by degeneracy pressure, and at the end of the chain, black holes. A diffuse gas of computronium has no obvious mechanism to resist its own gravity in the long run, it would presumably either settle into one of these familiar structures or collapse into a black hole. To settle into one of these familiar object could imply that computronium’s abundance is limited by physical constraints like temperature and pressure in the core of planets and stars. The black hole case is also interesting, because a black hole is theoretically the densest possible computer (Bekenstein), but everything it computes sits behind an event horizon, and recovering information from its Hawking radiation appears to require staggering amounts of computation in itself. So either way computronium could end up inhabiting—maybe in a weak proportion—the kinds of structures we already see, or hardly see (black holes).
Moreover, the Standard Model of cosmology tells us that most of the content of the universe is dark. Setting dark energy aside, there is a strong consensus that dark matter substantially outweighs ordinary matter. We don’t know what dark matter actually is, and we cannot rule out that it would make as good a computational harware as ordinary matter. We think dark matter interacts with ordinary matter only gravitationally (or, at most, very weakly through some other channel) but we don’t know whether dark matter interacts non-gravitationally with itself. If such self-interactions exist, they could plausibly be exploited for computation, much as we exploit electromagnetism in classical and quantum computers. If we accept agnosticism on this point, then, all else equal, the prior probability that computronium would be built out of dark matter is higher than that it would be built out of ordinary matter, simply because there is far more of it. We could already be inside a computronium bubble made of dark matter without realizing it.
Thanks. I would add that my first point—that we should stay humble about what a superintelligence ought to do—also extends to how it ought to do it.
Suppose superintelligences do converge on the most extreme solution : a computronium bubble expanding at light speed. That still doesn’t imply they would convert the entire content of their light cone into a uniform computational medium visible from parsecs away. Speaking as a non-physicist, my impression is that the structures we see in the universe are not arbitrary, they exist because they are stable equilibria. Matter at cosmological scales seems to end up in a fairly short list: planets and chemically bound bodies held together by electromagnetic forces, stars supported by nuclear fusion, neutron stars supported by degeneracy pressure, and at the end of the chain, black holes. A diffuse gas of computronium has no obvious mechanism to resist its own gravity in the long run, it would presumably either settle into one of these familiar structures or collapse into a black hole. To settle into one of these familiar object could imply that computronium’s abundance is limited by physical constraints like temperature and pressure in the core of planets and stars. The black hole case is also interesting, because a black hole is theoretically the densest possible computer (Bekenstein), but everything it computes sits behind an event horizon, and recovering information from its Hawking radiation appears to require staggering amounts of computation in itself. So either way computronium could end up inhabiting—maybe in a weak proportion—the kinds of structures we already see, or hardly see (black holes).
Moreover, the Standard Model of cosmology tells us that most of the content of the universe is dark. Setting dark energy aside, there is a strong consensus that dark matter substantially outweighs ordinary matter. We don’t know what dark matter actually is, and we cannot rule out that it would make as good a computational harware as ordinary matter. We think dark matter interacts with ordinary matter only gravitationally (or, at most, very weakly through some other channel) but we don’t know whether dark matter interacts non-gravitationally with itself. If such self-interactions exist, they could plausibly be exploited for computation, much as we exploit electromagnetism in classical and quantum computers. If we accept agnosticism on this point, then, all else equal, the prior probability that computronium would be built out of dark matter is higher than that it would be built out of ordinary matter, simply because there is far more of it. We could already be inside a computronium bubble made of dark matter without realizing it.