I don’t understand the argument. If the limit is above that needed for ASI, we’re dead. The fact that it could also be below or above the level of aligned ASI doesn’t change anything.
If you are suggesting that P(unethical ASI | random ASI created) = ~1, then yes, in that case, I agree, this does not matter.
If however, there is some probability of surviving a randomly created ASI, say 10%, then in this case, there is a difference between setting the administrative limit above and below it. If it is above, then we have a chance of surviving the creation of the AGI, but if the administrative limit is below , then we have no chance of surviving.
The problem is, we don’t actually know where is (or for that matter ). So we make a guess, and set an administrative limit. If we want to be safer, we ordinarily would think that lower administrative limits are safer, but based on the arguments in my post, I think there are times where lowering the administrative limit can make it less safe (if we lower the limit below but not below we would have been safer with the higher administrative limit).
I don’t understand the argument. If the limit is above that needed for ASI, we’re dead. The fact that it could also be below or above the level of aligned ASI doesn’t change anything.
If you are suggesting that P(unethical ASI | random ASI created) = ~1, then yes, in that case, I agree, this does not matter.
If however, there is some probability of surviving a randomly created ASI, say 10%, then in this case, there is a difference between setting the administrative limit above
The problem is, we don’t actually know where
But that’s the claim that’s obviously not correct. If we are drawing randomly from the distribution, the odds are infinitesimal.