IMO, there’s another major misprediction, and I’d argue that we don’t even need LLMs to make it a misprediction, and this is the prediction that within a few days/weeks/months we go from AI that was almost totally incapable of intellectual work to AI that can overpower humanity.
This comment also describes what I’m talking about:
(Yes, the Village Idiot to Einstein post also emphasized the vastness of the space above us, which is what Adam Scholl claimed and I basically agree with this claim, the issue is that there’s another claim that’s also being made).
The basic reason for this misprediction is as it turns out, human variability is pretty wide, and the fact that human brains are very similar is basically no evidence (I was being stupid about this in 2022):
And also, no domain has actually had a takeoff as fast as Eliezer Yudkowsky thought in either the Village Idiot to Einstein picture or his own predictions, but Ryan Greenblatt and David Matolcsi already made them, so I merely need to link them (1, 2, 3).
Also, a side note is that I disagree with Jacob Cannell’s post, and the reasons are that it’s not actually valid to compare brain FLOPs to computer FLOPs in the way Jacob Cannell does:
(Yes, I’m doing a lot of linking because other people have already done the work, I just want to share the work rather than redo things all over again).
@StanislavKrym I’m tagging you since I significantly edited the comment.
IMO, there’s another major misprediction, and I’d argue that we don’t even need LLMs to make it a misprediction, and this is the prediction that within a few days/weeks/months we go from AI that was almost totally incapable of intellectual work to AI that can overpower humanity.
This comment also describes what I’m talking about:
How takeoff used to be viewed as occuring in days, weeks or months from being a cow to being able to place ringworlds around stars:
(Yes, the Village Idiot to Einstein post also emphasized the vastness of the space above us, which is what Adam Scholl claimed and I basically agree with this claim, the issue is that there’s another claim that’s also being made).
The basic reason for this misprediction is as it turns out, human variability is pretty wide, and the fact that human brains are very similar is basically no evidence (I was being stupid about this in 2022):
The range of human intelligence is wide, actually.
And also, no domain has actually had a takeoff as fast as Eliezer Yudkowsky thought in either the Village Idiot to Einstein picture or his own predictions, but Ryan Greenblatt and David Matolcsi already made them, so I merely need to link them (1, 2, 3).
Also, a side note is that I disagree with Jacob Cannell’s post, and the reasons are that it’s not actually valid to compare brain FLOPs to computer FLOPs in the way Jacob Cannell does:
Why it’s not valid to compare brain FLOPs to computer FLOPs in the way Jacob Cannell does, part 1
Why it’s not valid to compare brain FLOPs to computer FLOPs in the way Jacob Cannell does, part 2
I generally expect it to be 4 OOMs at least better, which cashes out to at least 3e19 FLOPs per Joule:
The limits of chip progress/physical compute in a small area assuming we are limited to irreversible computation
(Yes, I’m doing a lot of linking because other people have already done the work, I just want to share the work rather than redo things all over again).
@StanislavKrym I’m tagging you since I significantly edited the comment.