Assessing Kurzweil predictions about 2019: the results

EDIT: Mean and stan­dard de­vi­a­tion of in­di­vi­di­d­ual pre­dic­tions can be found here.

Thanks to all my brave as­ses­sors, I now have the data about Kurzweil’s 1999 pre­dic­tions about 2019.

This was a fol­low up to a pre­vi­ous as­sess­ment about his pre­dic­tions about 2009, which showed a mixed bag. Roughly evenly di­vided be­tween right and wrong, which I found pretty good for ten-year pre­dic­tions:

So, did more time al­low for trends to over­come noise or more ways to go wrong? Pause for a mo­ment to cal­ibrate your ex­pec­ta­tions.

Meth­ods and thanks

So, for the 2019 pre­dic­tions, I di­vided them into 105 sep­a­rate state­ments, did a call for vol­un­teers, with in­struc­tions here; the main rele­vant point be­ing that I wanted their as­sess­ment for 2019, not for the (pos­si­bly tran­sient) cur­rent situ­a­tion. I got 46 vol­un­teers with valid email ad­dresses, of which 34 re­turned their pre­dic­tions. So many thanks, in re­verse alpha­bet­i­cal or­der, to Zvi Mow­show­itz, Zheng­dong Wang, Yann Riviere, Uriel Fiori, or­thonor­mal, Nuño Sem­pere, Nathan Ar­mishaw, Koen Holt­man, Kel­ler Scholl, Jaime Sevilla, Gareth McCaughan, Eli Rose and Dillon Plun­kett, Daniel Koko­ta­jlo, Anna Gar­diner… and oth­ers who have cho­sen to re­main anony­mous.

The results

Enough back­ground; what did the as­ses­sors find? Well, of the 34 as­ses­sors, 24 went the whole hog and did all 105 pre­dic­tions; on av­er­age, 91 pre­dic­tions were as­sessed by each per­son, a to­tal of 3078 in­di­vi­d­ual as­sess­ments[1].

So, did more time al­low for more per­spec­tive or more ways to go wrong? Well, Kurzweil’s pre­dic­tions for 2019 were con­sid­er­ably worse than those for 2009, with more than half strongly wrong:

In­ter­est­ing details

The (anonymised) data can be found here[2], and I en­courage peo­ple to down­load and as­sess it them­selves. But some in­ter­est­ing re­sults stood out to me:

Pre­dic­tor agreement

Tak­ing a sin­gle pre­dic­tion, for in­stance the first one:

  • 1: Com­put­ers are now largely in­visi­ble. They are em­bed­ded ev­ery­where—in walls, ta­bles, chairs, desks, cloth­ing, jew­elry, and bod­ies.

Then we can com­pute the stan­dard de­vi­a­tion of the pre­dic­tors’ an­swer for that pre­dic­tion. This gives an im­pres­sion of how much dis­agree­ment there was be­tween pre­dic­tors; in this case, it was 0.84.

Perfect agree­ment would be a stan­dard de­vi­a­tion of 0; max­i­mum dis­agree­ment (half find “1”, half find “5″) would be a stan­dard de­vi­a­tion of 2. Perfect spread—equal num­bers of 1s, 2s, 3s, 4s, and 5s—would have a stan­dard de­vi­a­tion of 1.4.

Across the 105 pre­dic­tions, the max­i­mum stan­dard de­vi­a­tion was 1.7, the min­i­mum was 0 (perfect agree­ment), and the av­er­age was 0.97. So the pre­dic­tors had a medium ten­dency to agree with each other.

Most agree­ment/​fals­est predictions

There was perfect agree­ment on five pre­dic­tions; and on all of these, the agreed pre­dic­tion was always “5”: “False”.

Th­ese pre­dic­tions were:

  • 51: “Phone” calls rou­tinely in­clude high-re­s­olu­tion three-di­men­sional images pro­jected through the di­rect-eye dis­plays and au­di­tory lenses.

  • 55: [...] Thus a per­son can be fooled as to whether or not an­other per­son is phys­i­cally pre­sent or is be­ing pro­jected through elec­tronic com­mu­ni­ca­tion.

  • 59: The all-en­velop­ing tac­tile en­vi­ron­ment is now widely available and fully con­vinc­ing.

  • 62: [...] Th­ese tech­nolo­gies are pop­u­lar for med­i­cal ex­am­i­na­tions, as well as sen­sual and sex­ual in­ter­ac­tions with other hu­man part­ners or simu­lated part­ners.

  • 63: [...] In fact, it is of­ten the preferred mode of in­ter­ac­tion, even when a hu­man part­ner is nearby, due to its abil­ity to en­hance both ex­pe­rience and safety.

As you can see, Kurzweil suffered a lot from his VR pre­dic­tions. This seems a peren­nial thing: Hol­ly­wood is always con­vinced that mass 3D is just around the cor­ner; tech­nol­o­gists are con­vinced that VR is im­mi­nent.

Most ac­cu­rate pre­dic­tion:

With a mean score of 1.3, the pre­dic­tion deemed most ac­cu­rate was:

  • 83: The ex­is­tence of the hu­man un­der­class con­tinues as an is­sue.

Now this might seem a triv­ial pre­dic­tion, es­pe­cially in ret­ro­spect, but I want to defend Kurzweil here—it was not at all cer­tain in 1999, with many utopian changes fore­seen and ex­pected, that this would still be an is­sue.

The next pre­dic­tion deemed most ac­cu­rate (mean of 1.4), is:

  • 82: Peo­ple at­tempt to pro­tect their pri­vacy with near-un­break­able en­cryp­tion tech­nolo­gies, but pri­vacy con­tinues to be a ma­jor poli­ti­cal and so­cial is­sue with each in­di­vi­d­ual’s prac­ti­cally ev­ery move stored in a database some­where.

This is truly non-triv­ial for 1999, and I do give Kurzweil credit for that.

Least agreement

With a stan­dard de­vi­a­tion of 1.7, the pre­dic­tors dis­agreed the most on this pre­dic­tion:

  • 37: Com­pu­ta­tion in gen­eral is ev­ery­where, so a stu­dent’s not hav­ing a com­puter is rarely an is­sue.

This may have to do with differ­ent judge­ment over the ex­tent of “ev­ery­where” and “rarely an is­sue”, or over who might or might not find this to be an is­sue.

The next pre­dic­tion with the most dis­agree­ment (st dev 1.6) is:

  • 16: Ro­tat­ing mem­o­ries and other elec­trome­chan­i­cal com­put­ing de­vices have been fully re­placed with elec­tronic de­vices.

It’s pos­si­ble that “fully” was a prob­lem here, but I see this pre­dic­tion as be­ing just false.

Most “Can­not De­cide”

This pre­dic­tion had more than of pre­dic­tors choos­ing “Can­not De­cide”:

  • 20: It is now fully rec­og­nized that the brain com­prises many spe­cial­ized re­gions, each with its own topol­ogy and ar­chi­tec­ture of in­terneu­ronal con­nec­tions.

Maybe the am­bi­guity in “fully rec­og­nized” made this hard to as­sess. Or maybe, as sug­gested in the com­ments, be­cause this doesn’t look much like a “pre­dic­tion”, but an ob­vi­ously true state­ment?

A ques­tion of timeline?

It’s been sug­gested that Kurzweil’s pre­dic­tions for 2009 are mostly cor­rect in 2019. If this is the case—Kurzweil gets the facts right, but the timeline wrong—it would be in­ter­est­ing to re­visit these pre­dic­tions in 2029 (if he is a decade op­ti­mistic) and 2039 (if he ex­pected things to go twice as fast). Though many of his pre­dic­tions seem to be of the type “once true, always true”, so his score should rise with time, as­sum­ing con­tin­u­ing tech­nolog­i­cal ad­vance and no dis­asters.

In conclusion

Again, thanks to all the vol­un­teers who as­sessed these pre­dic­tions and thanks to Kurzweil who, un­like most prog­nos­ti­ca­tors, had the guts and the cour­tesy to write down his pre­dic­tions and give them a date.

I strongly sus­pect that most peo­ple’s 1999 pre­dic­tions about 2019 would have been a lot worse.

  1. Five as­sess­ments of the 3078 re­turned ques­tion marks; I re­placed these with “3” (“Can­not De­cide”). Four as­ses­sors of the 34 left gaps in their pre­dic­tions, in­stead of work­ing through the ran­domly or­dered pre­dic­tions; to two sig­nifi­cant figures, ex­clud­ing these four didn’t change any­thing, so I in­cluded them all. ↩︎

  2. Each column is an in­di­vi­d­ual pre­dic­tor, each row an in­di­vi­d­ual pre­dic­tion. ↩︎