In the last similar thread someone pointed out that we’re just talking about increasing existential risk in the tiny zone were we observe (or reasonably extrapolate) each other existing, not the entire universe. It confuses the issue to talk about destruction of the universe.
Really this is all recursive to Joy’s “Grey goo” argument. I think what needs to be made explicit is weighing our existential risk if we do or don’t engage in a particular activity. And since we’re not constrained to binary choices, there’s no reason for that to be a starting point, unless it’s nontransparent propaganda to encourage selection of a particular unnuanced choice.
A ban on the production of all novel physics situations seems more extreme than necessary (although the best arguments for that should probably be heard and analyzed). But unregulated, unreviewed freedom to produce novel physics situations also seems like it would be a bit extreme. At the least, I’d like to see more analysis of the risks of not engaging in such experimentation. This stuff is probably very hard to get right, and at some point we’ll probably get it fatally wrong in one way or another and all die. But let’s play the long odds with all the strategy we can, because the alternative seems like a recursive end state (almost) no matter what we do.
In the last similar thread someone pointed out that we’re just talking about increasing existential risk in the tiny zone were we observe (or reasonably extrapolate) each other existing, not the entire universe. It confuses the issue to talk about destruction of the universe.
Really this is all recursive to Joy’s “Grey goo” argument. I think what needs to be made explicit is weighing our existential risk if we do or don’t engage in a particular activity. And since we’re not constrained to binary choices, there’s no reason for that to be a starting point, unless it’s nontransparent propaganda to encourage selection of a particular unnuanced choice.
A ban on the production of all novel physics situations seems more extreme than necessary (although the best arguments for that should probably be heard and analyzed). But unregulated, unreviewed freedom to produce novel physics situations also seems like it would be a bit extreme. At the least, I’d like to see more analysis of the risks of not engaging in such experimentation. This stuff is probably very hard to get right, and at some point we’ll probably get it fatally wrong in one way or another and all die. But let’s play the long odds with all the strategy we can, because the alternative seems like a recursive end state (almost) no matter what we do.