Simulation of technological progress (work in progress)

I’ve made a model/​simu­la­tion of tech­nolog­i­cal progress, that you can down­load and run on your lap­top.

My goal is to learn some­thing about in­tel­li­gence ex­plo­sions, take­off speeds, dis­con­ti­nu­ities, hu­man-level mile­stones, AGI vs. tools, bot­tle­necks, or some­thing else. I’ll be happy if I can learn some­thing about even one of these things, even if it’s just a minor up­date and not any­thing close to con­clu­sive.

So far I’ve just got a very ba­sic ver­sion of the model built. It works, but it’s cur­rently un­clear what—if any­thing—we can learn from it. I need to think more about whether the as­sump­tions it uses are re­al­is­tic, and I need to ex­plore the space of pa­ram­e­ter set­tings more sys­tem­at­i­cally.

I’m post­ing it here to get feed­back on the ba­sic idea, and maybe also on the model so far if peo­ple want to down­load it and play around. I’m par­tic­u­larly in­ter­ested in ev­i­dence/​ar­gu­ments about whether or not this is a pro­duc­tive use of my time, and ar­gu­ments that some hid­den as­sump­tion my model makes is prob­le­mat­i­cally de­ter­min­ing the re­sults.

If you want to try out the model your­self, down­load NetLogo here and then open the file in this folder.

How the model works:

The main part of the model con­sists of re­search pro­jects, which are lists of var­i­ous types of task. Civ­i­liza­tion com­pletes tasks to com­plete re­search pro­jects, and when pro­jects get finished, civ­i­liza­tion gets a “bonus” which al­lows it to do new types of task, and to do some old types faster.

The pro­jects, the lists of tasks needed to com­plete them, the speeds at which civ­i­liza­tion can do the tasks, and the bonuses granted by com­plet­ing pro­jects are all ran­domly gen­er­ated, typ­i­cally us­ing ex­po­nen­tial dis­tri­bu­tions and of­ten with pa­ram­e­ters you can change in the UI. Other im­por­tant pa­ram­e­ters can be changed in the UI also, such as how many task types are “off limits” for tech­nolog­i­cal im­prove­ment, and how many task types are “tem­porar­ily off limits” un­til some speci­fied level of tech­nol­ogy is reached.

As ex­plained so far, the model rep­re­sents bet­ter tech­nol­ogy lead­ing to more re­search di­rec­tions (more types of task be­come available) and faster progress (civ­i­liza­tion can do tasks in less time).

Pro­jects are dis­played as dots/​stars which flicker as work is done on them. When they com­plete, they turn into big green cir­cles. Their lo­ca­tion in the dis­play rep­re­sents how difficult they are to com­plete: the x-axis en­codes how many tasks are in­volved, and the y-axis en­codes how many differ­ent kinds of tasks are in­volved. To the left of the main dis­play is a graph that tracks a bunch of met­rics I’ve deemed in­ter­est­ing, scaled so that they all have similar heights.

There are sev­eral kinds of diminish­ing re­turns and sev­eral kinds of in­creas­ing re­turns in the model.

Diminish­ing:

  • “Pro­jects farther from the bot­tom left cor­ner on the dis­play re­quire ex­po­nen­tially more tasks and ex­po­nen­tially more kinds of tasks.

  • Pro­ject bonuses work as fol­lows: Pick a ran­dom bag of tasks, pick ran­dom speeds for each task, com­pare the re­sults to the cur­rent state-of-the-art speeds on those tasks and up­date the state of the art ac­cord­ingly. Thus, the faster civ­i­liza­tion already is at com­plet­ing a task, the more pro­jects will need to be com­pleted (on av­er­age) to im­prove speed at that task fur­ther.

  • Fi­nally, there is the usual “low-hang­ing fruit” effect where the pro­jects which can get done quickly do so, leav­ing harder and harder pro­jects re­main­ing.

In­creas­ing:

  • The more pro­jects you do, the more bonuses you get. This makes you faster at com­plet­ing pro­jects...

  • And opens up new pro­jects to work on, some of which will be low-hang­ing fruit.


The model also has a sim­ple mod­ule rep­re­sent­ing the “eco­nomic” side of things -- i.e. over time, civ­i­liza­tion can work on a greater num­ber of pro­jects si­mul­ta­neously, if you choose. I have a few differ­ent set­tings rep­re­sent­ing differ­ent sce­nar­ios:

    • “All pro­jects all the time” rep­re­sents a situ­a­tion where sci­ence has loads of fund­ing and/​or ex­cel­lent plan­ning of re­search paths, so that the only con­straints on finish­ing pro­jects are whether and how fast the tasks in­volved can be done.

    • “100 doable pro­jects” rep­re­sents a situ­a­tion with fixed re­search bud­get: 100 doable pro­jects are be­ing worked on at any given time.

    • Scal­ing effort with pro­jectscom­pleted rep­re­sents “learn­ing by do­ing” where the more pro­jects have been com­pleted so far, the more effort is in­vested in do­ing more pro­jects.

    • Scal­ing effort with ma­chine­taskspeed rep­re­sents a situ­a­tion where how much effort is de­voted to sci­ence is pro­por­tional to how ad­vanced to­day’s tech is on av­er­age.

The “info” tab of the NetLogo file ex­plains things in more de­tail, if you are in­ter­ested.

What tends to hap­pen when I run it:

The model tends to pro­duce progress (speci­fi­cally, in the met­ric of “pro­jects com­pleted—see the log plot) some­where be­tween ex­po­nen­tial and su­per­ex­po­nen­tial. Some­times it dis­plays what ap­pears to be a clear ex­po­nen­tial trend (a very straight line on the log scale) that fairly rapidly tran­si­tions into a sin­gu­lar­ity (a ver­ti­cal line on the log scale).

In­ter­est­ingly, progress in the met­ric “% of tasks done faster thanks to re­search” is not typ­i­cally ex­po­nen­tial, much less sin­gu­lar­ity; it is usu­ally a jumpy but more or less lin­ear march from 0 to 100.

Some­times progress stag­nates, though I’ve only seen this hap­pen ex­tremely early on—I’ve never seen steady ex­po­nen­tial growth fol­lowed by stag­na­tion.

For a while it seemed that progress would typ­i­cally shoot through the roof around the time that al­most all tasks were doable & be­ing im­proved. This is what Am­dahl’s Law would pre­dict, I think: Get rid of the last few bot­tle­necks and progress will soar. How­ever, I now think that’s wrong; the growth still hap­pens even if a sub­stan­tial frac­tion of tasks are “off-limits,” and/​or off-limits tem­porar­ily. I’m not sure what to think now, but af­ter I give my head a rest I ex­pect ideas will come.

The var­i­ous pa­ram­e­ter set­tings I’ve put into the model seem to have sur­pris­ingly lit­tle effect on all of the above. They af­fect how long ev­ery­thing takes but rarely do they af­fect the fun­da­men­tal shape of the tra­jec­tory. In par­tic­u­lar, re­mov­ing the “effort feed­back loop” en­tirely, by choos­ing “all pro­jects all the time” or “100 doable pro­jects” would (I pre­dicted) slow down progress a lot, but in prac­tice we still seem to get sin­gu­lar­i­ties. Of course, I haven’t sys­tem­at­i­cally com­pared the re­sults; this is just the vague im­pres­sion I get from the hand­ful of differ­ent runs I’ve done.


Doubts I have about the ac­cu­racy of the model & ideas for things to add

  • Most im­por­tantly, there are prob­a­bly ma­jor flawed as­sump­tions I’ve made in build­ing this model that I haven’t re­al­ized yet. I don’t know what I don’t know.

  • I worry that the re­sults de­pend in a brit­tle fash­ion on the prob­a­bil­ity dis­tri­bu­tions and ranges that I use to gen­er­ate the pro­jects. In other words, I’m wor­ried that the re­sults, though ro­bust to the pa­ram­e­ters I’ve put in the UI, are not ro­bust to hid­den as­sump­tions that I made hastily.

  • Often if you can’t do a pro­ject one way, there is an­other path via which you can do it. Thus we can e.g. use brute force search in a com­puter + a small amount of think­ing to re­place a larger amount of a differ­ent kind of think­ing. But in my model, all pro­jects have a sin­gle list of tasks that need to be done to com­plete the pro­ject. Is this a prob­lem?

  • Maybe I should try to build a ver­sion of this model that gets ex­po­nen­tial growth fol­lowed by stag­na­tion, or at least some sort of big s-curve? Maybe the rea­son I haven’t seen this be­hav­ior so far is that I haven’t put much effort into look­ing for it.

  • Ul­ti­mately I’d like to have a much richer econ­omy in the model, with differ­ent fac­tions buy­ing and sel­l­ing things as they sep­a­rately ad­vance up the tech tree. This sounds hard to im­ple­ment in code, but maybe it’s eas­ier than I think. Maybe eco­nomics can help me with this.

  • Cur­rently I have to wait a minute or so each run, de­pend­ing on set­tings (some runs hap­pen in just a few sec­onds). This is my very first cod­ing pro­ject so the code is prob­a­bly atro­cious and could run sig­nifi­cantly faster if I op­ti­mized it. If I could run it many times, I’d learn how much the re­sults vary from run to run due to ran­dom­ness.