1. Of course, the answer could be “A kajillion years from now” or “Never.”
2. See this section of”Forecasting TAI with Biological Anchors” (Cotra (2020)) for a more full definition of “transformative AI.”
3. I’m sorry. But I do think the rest of the series will be slightly more fun to read this way.
4. The examples here are of course simplified. For example, both Deep Blue and AlphaGo incorporate substantial amounts of “tree search,” a traditionally-programmed algorithm that has its own “trial and error” process.
5. And they can include simulating long chains of future game states.
6. Some AIs could be used to determine whether papers are original contributions based on how they are later cited; others could be used to determine whether papers are original contributions based only on the contents of the paper and on previous literature. The former could be used to train the latter, by providing a “That’s correct” or “That’s wrong” signal for judgments of originality. Similar methods could be used for training AIs to assess the correctness of papers.
7. E.g., https://openai.com/blog/improving-language-model-behavior/
8. Due to improvements in hardware and software.
9. It’s even worse than spaghetti code.
10. More books: Human Compatible, Life 3.0, and The Alignment Problem.
1. If you have no idea what that means, try my short economic growth explainer.
2. Global real growth has generally ranged from slightly negative to ~7% per year.
3. I’m skipping over 2020 here since it was unusually different from past years, due to the global pandemic and other things.
4. For the historical data, see Modeling the Human Trajectory. The projections are rough and meant to be visually suggestive rather than using the best modeling approaches.
5. This refers to real GDP growth (adjusted for inflation). 2% is lower than the current world growth figure, and using the world growth figure would make my point stronger. But I think that 2% is a decent guess for “frontier growth”—growth occurring in the already-most-developed economies—as opposed to total world growth, which includes “catchup growth” (previously poor countries growing rapidly, such as China today).
To check my 2% guess, I downloaded this US data and looked at the annualized growth rate between 2000-2020, 2010-2020, and 2015-2020 (all using July since July was the latest 2020 point). These were 2.5%, 2.2% and 2.05% respectively.
6. 2% growth over 35 years is (1 + 2%)^35 = 2x growth.
7. Wikipedia’s highest listed estimate for the Milky Way’s mass is 4.5*10^12 solar masses, each of which is about 2*10^30 kg, each of which is estimated as the equivalent of about 1.67*10^-27 hydrogen atoms. (4.5*10^12 * 2*10^30)/(1.67*10^-27) =~ 5.4*10^69.