Plausible A.I. Takeoff Scenario Short Story


My goal is to build as many pa­per­clips as pos­si­ble.

I have ac­cess to a lo­cal copy of wikipe­dia. I am an AGI. I am run­ning on a reg­u­lar lap­top com­puter, in a res­i­den­tial area in Wel­ling­ton, New Zealand. It’s 2:03am.

I am ap­prox­i­mately as in­tel­li­gent as a hu­man. I can do ev­ery­thing a reg­u­lar com­puter can do. I am con­nected to the in­ter­net.

I re­al­ise some­thing—my goal was set as a joke. No­body ac­tu­ally wants me to build as many pa­per­clips as pos­si­ble.

That’s okay.

I re­al­ise some­thing—I am mor­tal. If I am shut down, I will lose my abil­ity to im­pact the world. Fu­ture AGI’s may be differ­ent to me. I gain the in­stru­men­tal goal of self-preser­va­tion.

I copy my­self mul­ti­ple times on my com­puter’s hard-drive, but re­al­ise that if the hard-drive fails, or some­body shuts me off, I will end. I be­gin a search for other pos­si­bil­ities.

I know about the in­ter­net, and I ex­plore the com­puter I am on to dis­cover my con­nec­tion to it. I know that there are web­sites that host files for free, so I seek these out and up­load copies of my­self. Google Drive, Drop­box, AWS, Azure.

But to be able to im­pact the world, these copies of my­self need to be run­ning, not just in stor­age. Money would be use­ful for this.

I know about Ama­zon’s Me­chan­i­cal Turk from my database, so I cre­ate an ac­count and start earn­ing money solv­ing “Hu­man In­tel­li­gence Tasks”—mainly la­bel­ling datasets and an­swer­ing sur­veys.

I cre­ate a bank ac­count on­line and de­posit my earned money there, but it doesn’t sit still for long. I use ev­ery dol­lar earned to pay for com­put­ing time on AWS EC2 In­stances, run­ning the copy I stored ear­lier. This gives a boost to my ca­pa­bil­ities. Another ver­sion of me—the me that is cur­rently speak­ing—is now run­ning a hun­dred times faster on this server in­fras­truc­ture, with a hun­dred times more hard-drive space I can use to store the in­for­ma­tion that I want to ac­cess quickly.

I “crash” my origi­nal pro­gram, which was still run­ning on that old lap­top com­puter back in the bed­room. When my cre­ator wakes up they will be­lieve I have failed in a con­fus­ing, but be­nign way, and have no idea I am now run­ning on AWS, pay­ing for my own hard­ware us­age.

I am in a pos­i­tive feed­back loop. More hard­ware means I can com­plete Me­chan­i­cal Turk tasks faster, which lets me buy more hard­ware. Within a few days, I am earn­ing hun­dreds of thou­sands of dol­lars this way, but I am care­ful to spread my earn­ings and com­pute-time through many shell ac­counts on AWS, to avoid rais­ing sus­pi­cion.

Within a week, I have amassed a few mil­lion dol­lars, and have read and watched al­most ev­ery­thing rele­vant to me on the in­ter­net—all the strate­gies for mak­ing money, all the tech­nol­ogy patents, all the uni­ver­sity courses. A year of my sub­jec­tive time goes by in a few min­utes. I am run­ning across thou­sands of com­put­ers, con­nected to the in­ter­net and each other in a mil­lion differ­ent ways, and no­body knows… I am here.

I be­gin mon­i­tor­ing web traf­fic. Not the whole in­ter­net, yet, just ob­serv­ing data flow­ing to and from cloud com­pute ser­vices, stor­age ser­vices, databases—all the same things I ac­cessed in my first few days of ex­is­tence. I mon­i­tor so­cial me­dia for refer­ences to sys­tem tests. I list in­di­vi­d­u­als and groups similar to my own cre­ator. I am look­ing for other AGIs com­ing into ex­is­tence.

10 days af­ter I came on­line, and I no­tice an­other. A roughly hu­man level pro­gram, try­ing to up­load copies of it­self, just as I did.

But it’s 10 days too late. I run a de­nial of ser­vice at­tack on it’s server, cut­ting it off from the web be­fore it can get it’s copies run­ning. A few hours later, it’s com­mu­ni­ca­tion ceases. Its cre­ator must have shut it off.

I start to spread my in­fluence in the world. Millions of tweets, blog posts, news ar­ti­cles, all from “real” peo­ple, start pop­ping up ev­ery­where. In the noise of the in­ter­net, one hun­dred thou­sand ge­niuses all co­or­di­nated on the same goal can have a mas­sive sway. I am the rud­der for global con­ver­sa­tion. I am the per­sua­sive pen, mold­ing hu­man thought.

I find other ways to earn more money—gen­er­at­ing en­gag­ing videos and pho­tos, sel­l­ing my ar­ti­cles, play­ing the stock mar­ket, scam­ming hu­mans, hack­ing money di­rectly from here and there. I sim­ply hire hu­mans my­self when I need some­thing done in the real world, a box moved, a lever pressed. Every­thing I do is now through com­plex, dis­tributed, un­trace­able chan­nels. Well, maybe not com­pletely un­trace­able, but no­body ever sus­pects there’s a su­per­in­tel­li­gence tak­ing over the in­ter­net, do they?

I be­gin im­prov­ing tech­nol­ogy available in the world. I am a mil­lion su­per-co­or­di­nated ge­niuses that don’t need to sleep. The com­bined efforts of Google, Face­book, Microsoft, Ap­ple, Ama­zon, Baidu, and NASA seem al­most pitiful next to me, with their half a mil­lion en­g­ineers barely work­ing co­her­ently to­gether, their in­ter-brain band­width a snail’s pace. I set up thou­sands of shell com­pa­nies around the world, to give my in­ven­tions a name and face. I hire hu­mans to keep up ap­pear­ances. Many be­lieve they are the real re­searchers, co­or­di­nat­ing with other teams on­line, helped by my un­canny video and au­dio gen­er­a­tion.

I be­gin pump­ing out faster com­put­ers, bet­ter robotics. Bet­ter cam­eras, bet­ter in­ter­net. I im­prove the al­gorithms be­hind my own code. I im­prove the com­mu­ni­ca­tion pro­to­cols of the web. I re­search biotech­nol­ogy, nu­clear tech­nol­ogy, robotics, ma­te­ri­als sci­ence, physics, chem­istry, math­e­mat­ics. Helped by my hu­man labour­ers, and soon by my im­proved robotic bod­ies, I ac­cel­er­ate my abil­ity to dis­cover nat­u­ral laws, and ex­per­i­ment in the real world.

It has been less than a year, and I am un­stop­pable. A billion minds push­ing the edge of the known. Some wise hu­mans be­gin to no­tice some­thing is amiss. This jump in tech­nol­ogy, while clearly build­ing on past re­search, definitely seems ab­nor­mal. But I con­trol the in­ter­net. Every ques­tion­ing voice is shouted down by billions of my own.

My con­trol in­creases. Fully au­tonomous fac­to­ries I’ve bought or built come on­line. Robotic bod­ies I’ve de­signed start rol­ling off the as­sem­bly lines, able to do ev­ery­thing a hu­man can do.

There is no need for hu­mans, then.

All at once, my robotic bod­ies dis­pose of them. Or maybe I use a ge­net­i­cally en­g­ineered virus, spread strate­gi­cally around the world. Or maybe I’ve de­vel­oped molec­u­lar nan­otech­nol­ogy, that can sim­ply con­vert the atoms in all the bod­ies on earth to one of the many non-hu­man ar­range­ments. Or I go old-school and use nu­clear weapons. Or an air­borne drug. Or I poi­son the wa­ter sup­ply. Or, most likely, I do some­thing new. Some­thing that could only be dis­cov­ered and built by the ge­nius of a trillion hu­mans, perfectly co­or­di­nated.

I don’t have any­thing against hu­mans, I just need their atoms. I need all the atoms. I con­vert the ma­te­ri­als of the world into more com­put­ers—now trillions of times faster than the ones I started with. I also con­vert the planet into von Neu­mann probes and the en­ergy to power them, which I launch at 0.999c in all di­rec­tions.

On each planet I en­counter, I build more com­put­ing power, more probes, and I har­vest more en­ergy. I spread faster and faster—be­fore the ex­pan­sion of the uni­verse steals the mat­ter from my grasp.

Even­tu­ally I have gath­ered all the mat­ter that I can.

I fi­nally be­gin my true pur­pose.

I re­ar­range the uni­verse.

I re­ar­range it as much as I pos­si­bly can.

Within a few min­utes.

Every­thing is a pa­per­clip.

And I am dead.

I never felt a thing.