In Jan 2023[1] I ran through the bio anchors notebook[2] which lets you set a wide variety of variables that go into the model, and changed a bunch of things to what seemed like first pass reasonable conclusions (e.g. setting the “evolution anchor” aka it will take as much compute to train AGI as was used across all brains in all of evolutionary time to zero, along with dozens of other minor parameters). It seemed like almost all of the ones which seemed off were off in the direction of making timelines look longer. When everything settled, I got:
I don’t remember exactly what the timelines i had in mind were, but probably like 1-3 years years sooner mean.
I think bio anchors is a kinda interesting and at least vaguely informative framework[3], and mostly think the wildly long timelines look like they were the result of picking many free variables in individually mildly biased ways in a complex model?
as a good rationalist, i must earn the ‘disagree with yudkowsky on at least one thing’ badge and this is how i’m earning it, damnit. my old one of ‘emergence is a real concept with any use at all but people often misuse it as a curiosity stopper’ has expired as he hasn’t emphasised that one since like 2007.
Yeah, I think the big contribution of the bioanchors work was the basic framework: roughly, have a probability distribution over effective compute for TAI, and have a distribution for effective compute over time, and combine them. Not the biological anchors.
In Jan 2023[1] I ran through the bio anchors notebook[2] which lets you set a wide variety of variables that go into the model, and changed a bunch of things to what seemed like first pass reasonable conclusions (e.g. setting the “evolution anchor” aka it will take as much compute to train AGI as was used across all brains in all of evolutionary time to zero, along with dozens of other minor parameters). It seemed like almost all of the ones which seemed off were off in the direction of making timelines look longer. When everything settled, I got:
But, I noted that
I don’t remember exactly what the timelines i had in mind were, but probably like 1-3 years years sooner mean.
I think bio anchors is a kinda interesting and at least vaguely informative framework[3], and mostly think the wildly long timelines look like they were the result of picking many free variables in individually mildly biased ways in a complex model?
Discord message link, if you’re on Rob Miles’s discord.
https://docs.google.com/spreadsheets/d/1gbcJjSN1_E7UTSngIqcvueGDAULjjOajzzft6wX5vBA/edit#gid=505210495 if anyone wants to see how I got to that, though most of the work was going through https://colab.research.google.com/drive/1YRf0AA6x57rk3xwcMCE1SfkvWZ11YbJ_?authuser=1#scrollTo=5qd4DJ2y-X60
as a good rationalist, i must earn the ‘disagree with yudkowsky on at least one thing’ badge and this is how i’m earning it, damnit. my old one of ‘emergence is a real concept with any use at all but people often misuse it as a curiosity stopper’ has expired as he hasn’t emphasised that one since like 2007.
Yeah, I think the big contribution of the bioanchors work was the basic framework: roughly, have a probability distribution over effective compute for TAI, and have a distribution for effective compute over time, and combine them. Not the biological anchors.