Lessons on AI Takeover from the conquistadors

(Talk given at an event on Sun­day 28th of June. Daniel Koko­ta­jlo is re­spon­si­ble for the talk, Ja­cob Lager­ros and David Lam­bert ed­ited the tran­script.

If you’re a cu­rated au­thor and in­ter­ested in giv­ing a 5-min talk, which will then be tran­scribed and ed­ited, sign up here.)

Daniel Koko­ta­jlo: I’m ba­si­cally go­ing to re­cap my con­quis­ta­dors posts. I’m in­ter­ested to hear what peo­ple think about it. I know a lot more about the situ­a­tion now be­cause I’ve been read­ing some his­tory books. But my over­all opinion hasn’t changed.

I wrote this post be­cause in talk­ing to var­i­ous peo­ple and read­ing some things, there was what seemed to me to be a sim­plis­tic model of how mil­i­tary con­flicts work, or how a mil­i­tary takeover would work. Here’s the sim­ple model.

If we’re imag­in­ing a sce­nario where an AI ap­pears, it will likely be in some lab some­where. So it will start off with very few re­sources, very small amounts of money, for ex­am­ple. So you would have to have some sort of god-like in­tel­li­gence or a tech­nolog­i­cal ad­van­tage in or­der to have more strength than the rest of the world com­bined.

I think the con­quis­ta­dors are coun­terex­am­ples to this. So for a brief overview, I picked out three con­quis­ta­dors. I had already known about Cortes and Pizarro, which were prob­a­bly the most ex­treme ex­am­ples. But I pre­dicted that there would be a gen­eral trend here, so I did some Googling and af­ter 20 min­utes I found Alfonso, who is an­other ex­am­ple of this trend.

Alfonso was a Por­tuguese ex­plorer gen­eral. Here is a map of his ex­plo­ra­tion.

Ben Pace: Can I ask what pe­riod of his­tory this was?

Daniel Koko­ta­jlo: This is all around the early 1500s. Alfonso was 1506 to 1513, and then Cortes was 1519 and Pizarro was 1530.

If you look at this map, ba­si­cally, Colum­bus went across the At­lantic in 1492, and the Por­tuguese were, in the same time, ex­plor­ing down the coast of Africa. They were all try­ing to get to China and In­dia’s lu­cra­tive trade routes. Ba­si­cally, within 30 to 50 years, the Span­ish had turned this en­tire re­gion into a Span­ish lake, con­trol­ling the main ports and only al­low­ing Span­ish ships.

The Por­tuguese did the same thing. Alfonso was re­spon­si­ble for the blue re­gion. It only took them six years or so, and in­volved defeat­ing the Ot­toman Em­pire in sev­eral naval bat­tles, con­quer­ing var­i­ous port cities around the edge, defeat­ing some In­dian em­pires, and so forth.

Here is the map of Por­tuguese ter­ri­tory in In­dia which blew my mind.

I didn’t re­al­ize that they had con­quered so much so quickly. I haven’t looked up this ex­actly, but it took many more years to con­quer all of this ter­ri­tory.

When I usu­ally think of colo­nial­ism in In­dia, I think of the Bri­tish. But ac­cord­ing to this map, the Por­tuguese made quite a lot of con­quests them­selves be­fore then. Any­way, the ad­van­tages were not god-like, in my opinion. I go into more de­tail in the post, but it was not like they had nanobots or any other su­per weapons in which there were no defenses against.

They also didn’t have ma­chine guns, or even nor­mal guns. They had ex­tremely an­cient ar­que­buses that took two min­utes to reload af­ter you fired them. Most of the time, they just used swords for their fight­ing.. I could go on.

I do think that they won be­cause of their tech­nol­ogy, but it definitely was not a god-like ad­van­tage. So I think this dis­rupts the sim­ple model that I men­tioned ear­lier. And it’s not like they just had more strength than the re­gions that they con­quered ei­ther.

Or if we do say that they had more strength than the re­gions that they con­quered, it’s not be­cause they had some sort of god-like ad­van­tage. Here’s my model which I think bet­ter de­scribes how these, and pos­si­bly many other, con­quests worked.

Mili­tary takeover is a diplo­matic pro­cess in which you con­vince peo­ple to obey you. You do that by offer­ing a bet­ter Schel­ling point than ri­val claimants to the throne, and by defeat­ing your ri­vals in bat­tle. You don’t need to defeat them all at once; you can start small and scale up.

Each vic­tory gets you more al­lies which you can de­ploy in the next bat­tle. Ri­vals can be­come al­lies af­ter, or even with­out, a fight. This is how these con­quests worked. It wasn’t like Pizarro showed up at the In­can Em­pire’s bor­ders and was met by a mas­sive army, which he then defeated. Same thing for Cortes. There were a se­ries of epi­sodes in which the con­quis­ta­dors in­cre­men­tally gained more power.

If there’s an AI takeover, maybe the AI will have god-like pow­ers with nanobots and other pow­er­ful tech­nolo­gies. But it is also en­tirely pos­si­ble that there could be an AI takeover with­out these things. Thank you.

Questions

Ben Pace: Thank you very much, Daniel. That was fas­ci­nat­ing and you crammed quite a lot into it. I have some ques­tions but so does ev­ery­one else. Daniel Filan, would you like to ask the first ques­tion?

Daniel Filan: Yes. This is a com­bi­na­tion of com­ment and ques­tion.

Think­ing about god-like pow­ers, I re­cently read a piece about Cortes tak­ing over the Aztec Em­pire, and ap­par­ently a re­ally big ad­van­tage was ar­mor im­per­vi­ous to the kinds of weapons that Aztecs had. If other peo­ple can’t hurt you, then it is, in a sense, a god-like power. What I am won­der­ing is to what ex­tent they had the god-like power of ratch­et­ing, where the con­quis­ta­dors couldn’t go back­wards or get hurt while still be­ing able to hurt the Aztecs.

Daniel Koko­ta­jlo: I think that fits, to some ex­tent, with the story of Cortes, which I’ve now read in great de­tail. I think that the post that you men­tioned, ex­ag­ger­ates the ex­tent to which the ar­mor was su­pe­rior.

For one thing, a lot of the Span­ish gave up their metal ar­mor and took up a Aztec cloth ar­mor in­stead be­cause it was more com­fortable and al­most as good, which sug­gests that their metal ar­mor wasn’t su­pe­rior. Also, plenty of con­quis­ta­dors were kil­led in bat­tle. I think that a lot of their suc­cess had to do with their tac­tics rather than just the ar­mor. That said, ar­mor did likely play a large part in their vic­tory. It is wrong, how­ever, to con­clude that they were in­vin­cible be­cause of it when in fact they came close to be­ing wiped out on sev­eral oc­ca­sions.

Ben Pace: John, would you like to ask the next ques­tion?

john­swent­worth: Let’s say, hy­po­thet­i­cally, that I’m plan­ning to take over the world through tech­nolog­i­cal means, what sort of tech­nol­ogy should I be fo­cus­ing on in or­der to get the most bang for my buck?

Daniel Koko­ta­jlo: I think that re­al­is­ti­cally, an AI takeover would mostly not be mil­i­tary.

john­swent­worth: I didn’t ask about AI takeover.

Daniel Koko­ta­jlo: Okay, so I have a list of tech­nolo­gies that I think would be the mod­ern equiv­a­lents of the ar­mor.

Maybe I’ll draft a blog post some­time about my own spec­u­la­tions on tech­nolo­gies that aren’t that far ahead into the fu­ture, but could be pretty pow­er­ful. That’s what I re­ally should have said in re­sponse to Daniel’s ques­tion. That yes, the ar­mor was re­ally con­se­quen­tial in some sense.

It en­abled the con­quis­ta­dors to go re­ally far in bat­tle and in poli­tics. But it wasn’t fun­da­men­tally differ­ent from what the Aztecs had. It was similar but bet­ter.

And similarly, I think that an AI de­sign­ing bet­ter guns, ar­mor, and co­or­di­na­tion tech­nolo­gies for sol­diers could end up be­ing ex­tremely ad­van­ta­geous mil­i­tar­ily, even if it’s not some­thing fun­da­men­tally new, like nanobots. Even if it is some­thing sim­ple like aim as­sist for sol­diers’ guns or steer­ing as­sist for ve­hi­cles, these could provide a mas­sive mil­i­tary ad­van­tage.

Ben Pace: Go on. Throw us a bone. Give us one ex­am­ple of a tech­nol­ogy, es­pe­cially a slightly more sur­pris­ing one. We’ve heard of guns.

Daniel Koko­ta­jlo: Sure. Aim as­sists for in­fantry rifles would be re­ally benefi­cial . The US mil­i­tary is cur­rently look­ing into this. But a sim­ple ver­sion could just be a cam­era hooked up to a gun and trig­ger so that when you ac­ti­vate it, it fires the bul­let when it calcu­lates that the bul­let will ac­tu­ally hit the tar­get.

You wouldn’t have to aim care­fully, you could just wave the gun in front of the tar­get and it shoots at ex­actly the right mo­ment.. A more so­phis­ti­cated ver­sion of this, which I have thought of, but not sure if any­one else has f, is hav­ing some sort of cold gas thruster on the tip of the gun.

Or maybe a gy­ro­scope al­low­ing the gun to move it­self al­though it sounds more com­pli­cated. This doesn’t sound like much, but I think it would prob­a­bly be a pretty big deal on the bat­tlefield. For one thing, you wouldn’t have to poke your head out of cover to shoot at the en­emy.

You could just poke the gun out and it would do the shoot­ing I also think be­ing sub­stan­tially more ac­cu­rate would make cover fire a lot more effec­tive. And that could al­low you to be much more mo­bile on the bat­tlefield. Any­how, this is just one ex­am­ple.

Ben Pace: It is at this point, I re­al­ize I’ve re­ally not ac­tu­ally tried to figure out how I would use mod­ern tech to im­prove mil­i­tary prac­tices. And I am a bit scared now. All right, the next ques­tioner is or­thonor­mal.

or­thonor­mal: Are there any known ex­am­ples of at­tempted con­quis­ta­dors that failed?

Daniel Koko­ta­jlo: That’s a re­ally good ques­tion. There’s prob­a­bly a half dozen to a dozen par­ties that wan­dered around the Amer­i­cas and didn’t re­ally find much and then died of star­va­tion or some­thing.

In fact, that very nearly hap­pens to Pizarro and Cortes. There weren’t any other ma­jor em­pires like the In­cas or the Aztecs in the Amer­i­cas. As for the rest of the world, I don’t know the full an­swer to that ques­tion. There were likely at­tempts to con­quer In­dia that failed be­fore even­tu­ally suc­ceed­ing.