BTW, a few days ago Eliezer made a specific prediction that is perhaps relevant to your discussion:
I [would very tentatively guess that] AGI to kill everyone before self-driving cars are commercialized
(I suppose Eliezer is talking about Level 5 autonomy cars here).
Maybe a bet like this could work:
At least one month will elapse after the first Level 5 autonomy car hits the road, without AGI killing everyone.
“Level 5 autonomy” could be further specified to avoid ambiguities. For example, like this:
The car must be publicly accessible (e.g. available for purchase, or as a taxi etc). The car should be able to drive from some East Coast city to some West Coast city by itself.
We might be able to falsify that in a few months.
There is a joint Google / OpenAI project called BIG-bench. They’ve crowdsourced ~200 of highly diverse text tasks (from answering scientific questions to predicting protein interacting sites to measuring self-awareness).
One of the goals of the project is to see how the performance on the tasks is changing with the model size, with the size ranging by many orders of magnitude.
A half-year ago, they presented some preliminary results. A quick summary:
if you increase the N of parameters from 10^7 to 10^10, the aggregate performance score grows roughly like log(N).
But after the 10^10 point, something interesting happens: the score starts growing much faster (~N).
And for some tasks, the plot looks like a hockey stick (a sudden change from ~0 to almost-human).
The paper with the full results is expected to be published in the next few months.
Judging by the preliminary results, the FOOM could start like this: