I don’t understand what point you’re trying to make. My point was that cooperative game theory doesn’t magically guarantee a UFAI will treat us nicely. It might work but only if there is a sufficiently substantial Everett branch with a FAI. The probability of that branch probably strongly depends on effort invested into FAI research.
If the probability of FAI (and friendly uploads etc) is near zero, then we’re doomed either way. But even though I believe the probability of provably friendly AI coming first is <50% , its definitely not 10^-11!
I don’t understand what point you’re trying to make. My point was that cooperative game theory doesn’t magically guarantee a UFAI will treat us nicely. It might work but only if there is a sufficiently substantial Everett branch with a FAI. The probability of that branch probably strongly depends on effort invested into FAI research.
If the probability of FAI (and friendly uploads etc) is near zero, then we’re doomed either way. But even though I believe the probability of provably friendly AI coming first is <50% , its definitely not 10^-11!
Fair enough, but there’s still an enormous incentive to work on FAI.
Of course, I was not trying to suggest otherwise.