The openphil contest is approaching, so I’m working on an edited version. Keeping this original version as-is seems like a good idea- both as a historical record and because there’s such a nice voiceover!
I’ve posted the current version over on manifund with a pdf version. If you aren’t familiar with manifund, I’d recommend poking around. Impact certificates are neat, and I’d like them to become more of a thing!
The main changes are:
Added a short section trying to tie together why the complexity argument actually matters.
Updated a few spots with notes for things that have happened between October 2022 and May 2023, including a section dedicated to NVIDIA quarterly reports.
A very large new section named The Redacted Section dedicated to [REDACTED].
Removed pretty much all risk-related arguments in favor of focusing on timelines to save some words. It is still very long, ack.
A bunch of small clarity/quality edits, including the removal of 704 unnecessary instances of the word “actually.”
Overall, I’m pretty happy with how the post has fared in the last several months. The largest miss is probably the revenue forecasts- I didn’t anticipate massive semiconductor export restrictions. Given the complexity, I’m not sure how to interpret this in terms of AI timelines yet. It’s notable that hyperscalers are a large and rapidly growing customer base for NVIDIA that already managed to mitigate temporary losses, and I doubt the recently strengthened race dynamics are going to change that (until those companies decide to push alternatives for ML hardware).
My timelines haven’t noticeably changed. GPT-4 is around the median of my previous vaguely-gut-defined capability distribution. I anticipate the next generation of applications that build some infrastructure around GPT-4 level systems (like the next version of github copilot) will surprise a few more people, just because the full capability of GPT-4 isn’t immediately apparent in a pure dialogue setting.
My P(doom) has actually decreased since I wrote the post: I’m down to around 30-35% ish. I had only recently gotten into serious technical safety research when I wrote the post, so some volatility isn’t surprising, but I’m glad it went the direction it did. That reduction is mostly related to some potential implications of predictor/simulator research efforts (not necessarily complete solutions, but rather certain things being easier than expected) and positive news about the nature of the problem and interpretability. (Worth noting that number expects Effort and I do not expect default flailing to work out, and that my estimate should still be treated as relatively volatile.)
The openphil contest is approaching, so I’m working on an edited version. Keeping this original version as-is seems like a good idea- both as a historical record and because there’s such a nice voiceover!
I’ve posted the current version over on manifund with a pdf version. If you aren’t familiar with manifund, I’d recommend poking around. Impact certificates are neat, and I’d like them to become more of a thing!
The main changes are:
Added a short section trying to tie together why the complexity argument actually matters.
Updated a few spots with notes for things that have happened between October 2022 and May 2023, including a section dedicated to NVIDIA quarterly reports.
A very large new section named The Redacted Section dedicated to [REDACTED].
Removed pretty much all risk-related arguments in favor of focusing on timelines to save some words. It is still very long, ack.
A bunch of small clarity/quality edits, including the removal of 704 unnecessary instances of the word “actually.”
Overall, I’m pretty happy with how the post has fared in the last several months. The largest miss is probably the revenue forecasts- I didn’t anticipate massive semiconductor export restrictions. Given the complexity, I’m not sure how to interpret this in terms of AI timelines yet. It’s notable that hyperscalers are a large and rapidly growing customer base for NVIDIA that already managed to mitigate temporary losses, and I doubt the recently strengthened race dynamics are going to change that (until those companies decide to push alternatives for ML hardware).
My timelines haven’t noticeably changed. GPT-4 is around the median of my previous vaguely-gut-defined capability distribution. I anticipate the next generation of applications that build some infrastructure around GPT-4 level systems (like the next version of github copilot) will surprise a few more people, just because the full capability of GPT-4 isn’t immediately apparent in a pure dialogue setting.
My P(doom) has actually decreased since I wrote the post: I’m down to around 30-35% ish. I had only recently gotten into serious technical safety research when I wrote the post, so some volatility isn’t surprising, but I’m glad it went the direction it did. That reduction is mostly related to some potential implications of predictor/simulator research efforts (not necessarily complete solutions, but rather certain things being easier than expected) and positive news about the nature of the problem and interpretability. (Worth noting that number expects Effort and I do not expect default flailing to work out, and that my estimate should still be treated as relatively volatile.)
Bit of a welp:
NVIDIA Q1 FY24 filings just came out. In the May 9th edit, I wrote:
In reality, it had already recovered and was in the process of setting a new record.