I may be wrong here but it sounds like Goran is proposing some limits on the power humans can exert on one another, not an all-encompassing nanny-bot.
I suppose if one were to push the boundaries far enough in any direction they would inevitably end up limited by the consequences of their actions for others but it seems likely that whatever boundaries such an AI set would be far beyond the boundaries already imposed on virtually all of us by limited knowledge, technology, morality etc.
I guess what I’m trying to say is plurality limits don’t seem all that concerning.
Which devolves to “I don’t want other human nations doing things I don’t like. I need an absolutely reliable tool I can use, something that won’t turn against me, that I can use against them”.
And so you end up with arms race, tool AGIs, and the reality that if one faction gains a sufficiently large advantage they will use their tools against others.
All these “we need to slow down AGI” calls are actually saying “losing is better than winning...”
That’s suicide for humans. See above.
Why would an AI which has been given the role of defending plurality start to think that paper clips are much more fun?
Because we have given up all autonomy and have no purpose in existing.
I may be wrong here but it sounds like Goran is proposing some limits on the power humans can exert on one another, not an all-encompassing nanny-bot.
I suppose if one were to push the boundaries far enough in any direction they would inevitably end up limited by the consequences of their actions for others but it seems likely that whatever boundaries such an AI set would be far beyond the boundaries already imposed on virtually all of us by limited knowledge, technology, morality etc.
I guess what I’m trying to say is plurality limits don’t seem all that concerning.
Which devolves to “I don’t want other human nations doing things I don’t like. I need an absolutely reliable tool I can use, something that won’t turn against me, that I can use against them”.
And so you end up with arms race, tool AGIs, and the reality that if one faction gains a sufficiently large advantage they will use their tools against others.
All these “we need to slow down AGI” calls are actually saying “losing is better than winning...”