Yeah, if the paperclipper values a paperclip today more than a paperclip tomorrow, then I suppose it will lose out to other AIs that have a lower time discounting rate and can delay gratification for longer. Unless these other AIs also use time discounting, e.g. the power-hungry AI could value a 25% chance of ultimate power today the same as a 50% chance tomorrow.
But then again, such contests can happen only if the two AIs arise almost simultaneously. If one of them has a head start, it will try to eliminate potential competition quickly, because that’s the utility-maximizing thing to do.
I suppose that’s the main reason to be pessimistic about FAI. It’s not just that FAI is more constrained in its actions, it also takes longer to build, and a few days’ head start is enough for UAI to win.
Yeah, if the paperclipper values a paperclip today more than a paperclip tomorrow, then I suppose it will lose out to other AIs that have a lower time discounting rate and can delay gratification for longer. Unless these other AIs also use time discounting, e.g. the power-hungry AI could value a 25% chance of ultimate power today the same as a 50% chance tomorrow.
But then again, such contests can happen only if the two AIs arise almost simultaneously. If one of them has a head start, it will try to eliminate potential competition quickly, because that’s the utility-maximizing thing to do.
I suppose that’s the main reason to be pessimistic about FAI. It’s not just that FAI is more constrained in its actions, it also takes longer to build, and a few days’ head start is enough for UAI to win.