It just seems very clear to me that the sort of person who is taken in by [Paul Christiano’s slow takeoff] essay is the same sort of person who gets taken in by Hanson’s arguments in 2008 and gets caught flatfooted by AlphaGo and GPT-3 and AlphaFold 2.
We can very loosely test this hypothesis by asking whether predictors on Metaculus were surprised by these developments, since Metaculus tends to generally agree with Paul Christiano’s model (see here for example).
Unfortunately, we can’t make many inferences with the data available, as it’s too sparse. Still, I’m leaving the following information here in case people find it interesting.
AlphaGo. There were two questions on Metaculus about Go before AlphaGo beat Lee Sedol. The first was this question about whether an AI would beat a top human Go player in 2016. Before AlphaGo became widely known—following the announcement of its match against Fan Hui—the median prediction was around 30%. After the announcement, the probability shot up to 90%. Unfortunately, this can’t be taken to be much evidence that Metaculus impressively foresaw a breakthrough that year, since Demis Hassabis had already hinted at a breakthrough at the time the question was opened. (Before the matches, Metaculus put the chances of AlphaGo beating Lee Sedol at 64%).
GPT-3. It’s unclear what relevant metrics would have counted as “predicting GPT-3”. There was a question for the best Penn Treebank perplexity score in 2019 and it turned out Metaculus over-predicted progress (though this was mostly a failure in operationalization, see Daniel Filan’s post-mortem). Metaculus had generally anticipated a great increase in parameter counts for ML models in early 2020, as evidenced by this question. More generally, GPT-3 doesn’t seem like a good example of a discontinuity in machine learning progress in perplexity when looking at the benchmark data. It’s possible GPT-3 is a discontinuity from previous progress in some other, harder to measure sense, but I’m not currently aware of what that might be.
AlphaFold 2. Metaculus wasn’t generally very surprised by a breakthrough in protein folding prediction. Since early 2019, predictors placed a greater than 80% chance that a breakthrough would happen by 2031 (note, AlphaFold 2 technically doesn’t count as a “breakthrough” by the strict definition in the question criteria). However, it is probably true that Metaculus was surprised that it happened so early.
Is GPT-3 perhaps some sort of discontinuity for how single-language text generation w/ neural networks is monetized? Have there been other companies that sold text completion as a service, metered out per token, before GPT-3?
Obviously, this isn’t a purely technical discontinuity, but I haven’t heard of any companies monetizing language models in this way in the past.
We can very loosely test this hypothesis by asking whether predictors on Metaculus were surprised by these developments, since Metaculus tends to generally agree with Paul Christiano’s model (see here for example).
Unfortunately, we can’t make many inferences with the data available, as it’s too sparse. Still, I’m leaving the following information here in case people find it interesting.
AlphaGo. There were two questions on Metaculus about Go before AlphaGo beat Lee Sedol. The first was this question about whether an AI would beat a top human Go player in 2016. Before AlphaGo became widely known—following the announcement of its match against Fan Hui—the median prediction was around 30%. After the announcement, the probability shot up to 90%. Unfortunately, this can’t be taken to be much evidence that Metaculus impressively foresaw a breakthrough that year, since Demis Hassabis had already hinted at a breakthrough at the time the question was opened. (Before the matches, Metaculus put the chances of AlphaGo beating Lee Sedol at 64%).
GPT-3. It’s unclear what relevant metrics would have counted as “predicting GPT-3”. There was a question for the best Penn Treebank perplexity score in 2019 and it turned out Metaculus over-predicted progress (though this was mostly a failure in operationalization, see Daniel Filan’s post-mortem). Metaculus had generally anticipated a great increase in parameter counts for ML models in early 2020, as evidenced by this question. More generally, GPT-3 doesn’t seem like a good example of a discontinuity in machine learning progress in perplexity when looking at the benchmark data. It’s possible GPT-3 is a discontinuity from previous progress in some other, harder to measure sense, but I’m not currently aware of what that might be.
AlphaFold 2. Metaculus wasn’t generally very surprised by a breakthrough in protein folding prediction. Since early 2019, predictors placed a greater than 80% chance that a breakthrough would happen by 2031 (note, AlphaFold 2 technically doesn’t count as a “breakthrough” by the strict definition in the question criteria). However, it is probably true that Metaculus was surprised that it happened so early.
Is GPT-3 perhaps some sort of discontinuity for how single-language text generation w/ neural networks is monetized? Have there been other companies that sold text completion as a service, metered out per token, before GPT-3?
Obviously, this isn’t a purely technical discontinuity, but I haven’t heard of any companies monetizing language models in this way in the past.
EDIT: see also Gwern’s comment for why Penn Tree Bank Perplexity isn’t a good metric for discontinuities in language models. (https://www.lesswrong.com/posts/vwLxd6hhFvPbvKmBH/yudkowsky-and-christiano-discuss-takeoff-speeds?commentId=mKgEsfShs2xtaWz4K)