Does anyone know Yudkowsky’s elaboration on ASIs not doing combat against each other?
hereforYud
Karma: 0
I’m not sure I’d agree with your definition of success in trial runs. First, because the average response to how we handled COVID won’t generalize to viruses that are substantially more dangerous. If AI is paused and development continues, there will almost necessarily be larger risk in the larger capacity. If it’s substantially stronger than it was when it was initially paused, then it’s like comparing the COVID response to a hypothetical virus that’s visually much more dangerous—if it looked like the black death and there were videos of people passing out on the street because of it, I’m certain there would be a more prompt response.
Second, Operation Warp Speed was incredibly effective to my understanding in creating effective vaccines extremely quickly and then mass producing and distributing hundreds of millions of them. Using OWS as a model to improve from for the next pandemic seems like it could only help us.
I think it’s hard to make a list of when to tell that doing something once is helpful for doing them again, but some broad aspects would be A, the ability to learn from the first event, B lasting infrastructure or means of doing something as a result of the first event, C how well we’re able to test/understand A and B, the list goes on but I don’t think expanding it would necessarily be helpful
With your original comment, are you just saying that trial runs being generally helpful doesn’t always apply or are you saying that it wouldn’t apply in this instance and COVID is a strong indicator of that? If it’s the latter, can you give more reasons that the analogy works?
Well what would you say it depends on? Part of the COVID response was that we got lucky and the virus wasn’t as bad as it could have been. Had it been worse, the general response could have been better.
Here, it seems like the public sentiment is that AI is a net negative and that reacting to it would be a good option. If a good portion of the public agrees that we’ve pulled something much more dangerous than COVID out of the urn, how could this be analogous?
And then there’s the fact that if we have to Pause again, AI will almost necessarily be more dangerous. If post-COVID we know that the next virus will be worse, and potentially much worse, then COVID probably did offer some good degree of prep.
There are a lot of scenarios where that makes sense, but the breadth of situations we’re talking about is ridiculously wide. If 2 ASIs are made somewhat simultaneously 6 months from now, wouldn’t the first one to exist take a lot of precautions against the emergence of a 2nd ASI and prepare to end it immediately? It seems like the resources needed to calculate through peace and then continuously check that it’s being maintained from now to forever could very plausibly be greater than the resources needed to just end the other mind.
I definitely agree that ASI could keep the peace longer than humans have tended to, but even then, there could be weird motives in the ASI that encourage violence against things like it.