First, I want to clarify that I feel we’re going into a more interesting place, where there’s a better chance that you might find a point that invalidates Yudkowsky’s argument, and can thus convince him of the value of the model.
But it’s also important to realize that IMO, Yudkowsky is not just saying that biological anchors are bad. The more general problem (which is also developed in this post) is that predicting the Future is really hard. In his own model of AGI timelines, the factor that is basically impossible to predict until you can make AGI is the “how much resources are needed to build AGI”.
So saying “let’s just throw away the biological anchors” doesn’t evade the general counterargument that to predict timelines at all, you need to find information on “how much resources are needed to build AGI”, and that is incredibly hard. If you or Ajeya can argue for actual evidence in that last question, then yeah, I expect Yudkowsky would possibly update on the validity of the timeline estimates.
But at the moment, in this thread, I see no argument like that.
First, I want to clarify that I feel we’re going into a more interesting place, where there’s a better chance that you might find a point that invalidates Yudkowsky’s argument, and can thus convince him of the value of the model.
But it’s also important to realize that IMO, Yudkowsky is not just saying that biological anchors are bad. The more general problem (which is also developed in this post) is that predicting the Future is really hard. In his own model of AGI timelines, the factor that is basically impossible to predict until you can make AGI is the “how much resources are needed to build AGI”.
So saying “let’s just throw away the biological anchors” doesn’t evade the general counterargument that to predict timelines at all, you need to find information on “how much resources are needed to build AGI”, and that is incredibly hard. If you or Ajeya can argue for actual evidence in that last question, then yeah, I expect Yudkowsky would possibly update on the validity of the timeline estimates.
But at the moment, in this thread, I see no argument like that.