Maybe this is naive of me but why would you not just do the standard act-utilitarian thing? Having all of future scientific knowledge before intelligence augmentation is worth let’s say a 10% chance of destroying the world right now, future physics knowledge is 10% of total future knowledge, knowledge from the LHC is 1% of future physics knowledge, so to justify running it the probability of it destroying the world has to be less than 10^-4. The probability of an LHC black hole eating the world is the probability that LHC will create micro black holes, times the probability that they won’t Hawking-radiate away or decay in some other way, times the probability that if they survive they eat the Earth fast enough to be a serious problem, which product does indeed work out to much less than 10^-4 for reasonable probability assignments given available information including the new Mangano/Giddings argument. Repeat analysis for other failure scenarios and put some probability on unknown unknowns (easier said than done but unavoidable). Feel free to argue about the numbers.
Maybe this is naive of me but why would you not just do the standard act-utilitarian thing? Having all of future scientific knowledge before intelligence augmentation is worth let’s say a 10% chance of destroying the world right now, future physics knowledge is 10% of total future knowledge, knowledge from the LHC is 1% of future physics knowledge, so to justify running it the probability of it destroying the world has to be less than 10^-4. The probability of an LHC black hole eating the world is the probability that LHC will create micro black holes, times the probability that they won’t Hawking-radiate away or decay in some other way, times the probability that if they survive they eat the Earth fast enough to be a serious problem, which product does indeed work out to much less than 10^-4 for reasonable probability assignments given available information including the new Mangano/Giddings argument. Repeat analysis for other failure scenarios and put some probability on unknown unknowns (easier said than done but unavoidable). Feel free to argue about the numbers.