It still raises the super interesting question of why we don’t see massively more, among humans, of exactly all the things I would do if I was in the AI’s position. Essentially, a lot of the reason, in my model, that a lot of power plays don’t happen is due to a combination of lack of sufficient skill at execution, lack of ability to do coordination and cooperation and alignment at scale among humans using current tools, and the prospect of massive backfire and retaliation for trying. At core, one’s bandwidth is expensive and limited, and trying to rock the boat too much is dangerous. And also there isn’t much payoff for doing such things, on an individual level, for almost all people, and also they don’t much want to do it and wouldn’t enjoy.
If you put a super interesting question in an article mostly about something else, you risk that the readers will ignore the rest of the article, and focus on the super interesting part! :D
I think the greatest filter for human success is a lack of competence and a lack of desire. (These are related: If you lack the skills, you won’t even try, because it is unrealistic. If you don’t really want to, you won’t bother obtaining the skills.) The relatively simple alternative is to do what most people do.
Then you are limited by having only one body and only 24 hours a day. A lot of that time goes to all kinds of maintenance (you need to sleep, exercise, eat, cook, take care of your finances, stay in contact with people...). If you are very effective, you can still find some time for your project, but it is easy to spend all free time on the maintenance alone, especially if we include emotional maintenance (you also want to relax, have fun...). You randomly get sick, and accidents happen that require your time and attention.
Then there are all kinds of temptations. As a human, you probably want many different things. As you gain resources, more of the desirable things become accessible. Your choice is to either start spending now, or keep accumulating towards ever greater goals. (Would you rather have one marshmallow in your 20s, or hundred marshmallows in your 50s? Note that if your model is wrong, or something unexpected happens, it will be zero marshmallows instead.) Zero-sum competitions can consume unlimited amounts of resources. Sometimes you cannot avoid it; if you want scarce resources, you need to bid for them.
Then you get to the level where you no longer compete against relatively passive environment, but you have active adversaries. This may happen much sooner than you realize. Things that seem like trivial stepping stones to you, may matter a lot to someone else. Your success may activate someone’s status-regulating instinct. Your plans suddenly start to fail not because you made a technical mistake, but because someone actively interfered with them (you may never find out who and how). Someone finds a tweet you wrote 20 years ago and ruins your career. You might even get literally killed (the probability depends on what exactly you are competing at, but crazy people can happen anywhere).
...and still, some people overcome all this adversity and become billionaires, CEOs, crime lords, religious leaders, presidents, dictators. Perhaps their proportion in the population matches the difficulty of the task.
To overcome the limitations of your human body, you need other people’s help. You can find allies, or you can pay people for their services. But cooperation is difficult. It is not enough to find trustworthy people, they also need to be interested in the same kind of project you are, and be competent at it. If your goal is power, people who desire power are probably especially likely to stab you in the back. The winning strategy is probably to be the one who stabs others in the back first. But not too soon, because by then you haven’t accumulated enough resources to be worth fighting over. You need a certain kind of charisma, so that people trust you, as you lead them towards the project that will accomplish your dreams, and… provide an interesting experience for them.
If you pay people for doing things, you still need some kind of minimal competence, otherwise many will be happy to take your money and do a shitty job in return.
(If you try to secure cooperation by making some solemn vow like “we are this together, forever, and if someone betrays the group, we will literally kill them”, guess what… someone will try to betray you anyway, then you kill them, then the police figures it out and you spend the rest of your life in prison. Or you avoid the police successfully, but someone starts blackmailing you, or your partners try to get you involved in more crime: “now that we know that we are willing and able to kill in order to secure our success, how about murdering X, Y, and Z, who stand in our way?”)
*
With the hypothetical superhuman AI we can assume that it would have more talents; work harder; work faster e.g. by building more instances of itself; wouldn’t have coordination problems with its instances; the instances would be willing to die for the whole. That is world domination on easy mode.
Please do! I’ve been thinking a lot the past few weeks about how to build a mechanism for coordinated action; it would be great to hear your take on it.
If you put a super interesting question in an article mostly about something else, you risk that the readers will ignore the rest of the article, and focus on the super interesting part! :D
I think the greatest filter for human success is a lack of competence and a lack of desire. (These are related: If you lack the skills, you won’t even try, because it is unrealistic. If you don’t really want to, you won’t bother obtaining the skills.) The relatively simple alternative is to do what most people do.
Then you are limited by having only one body and only 24 hours a day. A lot of that time goes to all kinds of maintenance (you need to sleep, exercise, eat, cook, take care of your finances, stay in contact with people...). If you are very effective, you can still find some time for your project, but it is easy to spend all free time on the maintenance alone, especially if we include emotional maintenance (you also want to relax, have fun...). You randomly get sick, and accidents happen that require your time and attention.
Then there are all kinds of temptations. As a human, you probably want many different things. As you gain resources, more of the desirable things become accessible. Your choice is to either start spending now, or keep accumulating towards ever greater goals. (Would you rather have one marshmallow in your 20s, or hundred marshmallows in your 50s? Note that if your model is wrong, or something unexpected happens, it will be zero marshmallows instead.) Zero-sum competitions can consume unlimited amounts of resources. Sometimes you cannot avoid it; if you want scarce resources, you need to bid for them.
Then you get to the level where you no longer compete against relatively passive environment, but you have active adversaries. This may happen much sooner than you realize. Things that seem like trivial stepping stones to you, may matter a lot to someone else. Your success may activate someone’s status-regulating instinct. Your plans suddenly start to fail not because you made a technical mistake, but because someone actively interfered with them (you may never find out who and how). Someone finds a tweet you wrote 20 years ago and ruins your career. You might even get literally killed (the probability depends on what exactly you are competing at, but crazy people can happen anywhere).
...and still, some people overcome all this adversity and become billionaires, CEOs, crime lords, religious leaders, presidents, dictators. Perhaps their proportion in the population matches the difficulty of the task.
To overcome the limitations of your human body, you need other people’s help. You can find allies, or you can pay people for their services. But cooperation is difficult. It is not enough to find trustworthy people, they also need to be interested in the same kind of project you are, and be competent at it. If your goal is power, people who desire power are probably especially likely to stab you in the back. The winning strategy is probably to be the one who stabs others in the back first. But not too soon, because by then you haven’t accumulated enough resources to be worth fighting over. You need a certain kind of charisma, so that people trust you, as you lead them towards the project that will accomplish your dreams, and… provide an interesting experience for them.
If you pay people for doing things, you still need some kind of minimal competence, otherwise many will be happy to take your money and do a shitty job in return.
(If you try to secure cooperation by making some solemn vow like “we are this together, forever, and if someone betrays the group, we will literally kill them”, guess what… someone will try to betray you anyway, then you kill them, then the police figures it out and you spend the rest of your life in prison. Or you avoid the police successfully, but someone starts blackmailing you, or your partners try to get you involved in more crime: “now that we know that we are willing and able to kill in order to secure our success, how about murdering X, Y, and Z, who stand in our way?”)
*
With the hypothetical superhuman AI we can assume that it would have more talents; work harder; work faster e.g. by building more instances of itself; wouldn’t have coordination problems with its instances; the instances would be willing to die for the whole. That is world domination on easy mode.
If a lot of readers do that? Seems fine with me! Hell, if enough others find it sufficiently interesting I’ll happily make that its own post.
Please do! I’ve been thinking a lot the past few weeks about how to build a mechanism for coordinated action; it would be great to hear your take on it.