The AI-box for hunter-gatherers
The following is a minor curiosity that occurred to me regarding real-world analogies to the AI-box concept.
Fundamentally, the reason that we fear a randomly-chosen super-intelligent AI is twofold:
It would be smarter than us, so it could outwit us no matter what its goals.
We have no reason to expect its goals to exactly coincide with ours, so we expect its actions to be detrimental to us.
Now, let’s say you’re playing a game with a wide range of outcomes, over which you have a well-defined utility function. Let’s say you can choose a partner from some pool of potential partners, each with a random utility function. Let’s say the intelligence of each player is known to you. It would be unwise to choose a player more intelligent than yourself because
They could outwit you.
You have no reason to expect their goals to coincide with yours.
On the other hand, if you pick a less intelligent player then perhaps you could trick them into furthering your own goals, or at least ascertain whether their motives coincide with yours. At the very least you would be able to keep them from subverting your own goal-directed actions.
I conjecture that this explains why people sometimes fear those more intelligent than themselves, and also why people sometimes act dumb. Imagine a hunter-gatherer group considering inviting an outsider to join them. If the outsider’s motives are uncertain then the more intelligent the outsider, the less well-advised the group would be to let em in.
In fact, a situation analogous to an AI-box could arise:
Group member 1: We should let this intelligent outsider in, but we should keep tabs on them should they act against us.
Group member 2: But the fact that they’re more intelligent than us means that you *can’t* expect to keep tabs on them.
Group member 1: But couldn’t we set a test of their motives and only allow them in if they proves to have motives coinciding with ours?
In the case of people however group member 2 is often incorrect because the intelligence difference is small enough that some level of judgement is often possible. Unlike with a super intelligent AI where it is currently not even known how to judge its level of intelligence because it can self improve in short order.
Fair point, the analogy I made is a bit of a stretch, really.
Still, I think that uncertainty about “boxing” increases rapidly enough with intelligence that these considerations are significant even for intelligence differences that we observe between humans.
I agree that often people do not intuitively find it horrible scary to let an AI join their group aka let the AI out of the box, but on the other hand intuitively are wary of letting intelligent people* join their group for the reasons you listed above. In general people should be wary of both and more so for the AI since it does not share a common genetic history, brain structure, biological needs, and therefore and in general is a greater unknown. This lack of intuitive fear is much like how many species of animal lacked any fear of humans on first encounter like dodos.
Guns, germs, and steel: the fates of human societies By Jared M. Diamond
Lack of intuitive fear can cause instinction.
*Where intelligent people are the people more intelligent then the group by a good degree but not intelligent enough to bridge the inferential divide.
Well, if we’re talking about real world analogies to the AI box test, I have a minor caveat: sometimes, on Less Wrong, I see what seems to me to be the implied message that the more intelligent not only have an advantage over the less intelligent, but that the more intelligent can ipso facto completely control the less intelligent, at least in the context of hypotheticals and puzzles. This may be a wise assumption to make when we’re dealing with a self-improving AGI, or with Eliezer in the context of his famous tests. But in my own experience, I personally find it difficult to control some minds that are on some level weaker than my own. Think of training cats, or calming down a screaming toddler.
I also suspect that, without too much trouble, I could go to the seedier sides of any big city or a sleazy traveling carnival and find a fair number of people who might not have anything like my academic credentials, but who would be able to con me out of my money if I were foolish enough to listen to them. Is that different from playing the AI box game with Eliezer? I don’t know, because we don’t have transcripts of the two games.
Who here would be confident in his or her ability to win the AI box game against an experienced professional grifter of average intelligence cast in the AI role? For that matter, if such a game could be arranged, who—if cast in the role of the AI—would be confident in his or her ability to win the AI box game against a cranky toddler?
Any smart person who really knows how to control the actions of less intelligent people could potentially make a fortune advising corrections facilities, juvenile halls, and schools with severe chronic discipline problems.
I’m in general agreement with your post, but being good at X is quite different from being good at teaching how to do X.
Good point. Anyway, I think we agree that controlling people is not always an easy task, even for people who are very, very smart in other ways.
This assumes that we are talking about a single linear measure of intelligence, which doesn’t seem to be the case with normal humans. For example the same person can be above average in spacial reasoning but below average in verbal reasoning.
The relevant analogy for this would be social intelligence, so the person you should be the most suspicious of is the person who has displayed the greatest ability to manipulate social situations, though ironically if they are socially intelligent they should be able to prevent you suspecting them.