Yes, thanks! I mostly agree with that assessment,* though as an aside I have a beef with the implication that Bostrom, Yudkowsky, etc. expect discontinuities. That beef is with Paul Christiano, not you. :)
So far the biggest update this has been for me, I think, is that it seems to have shown that it’s quite possible to get an intelligence explosion even without economic feedback loops. Like, even with a fixed compute/money budget—or even with a fixed number of scientists and fixed amount of research funding—we could get singularity. At least in principle. This is weird because in practice I am pretty sure I remember reading that the growth we’ve seen so far can be best explained via an economic feedback loop: Better technology allows for bigger population and economy which allows for more scientists and funding which allows for better technology. So I’m a bit confused, I must say—my model is giving me results I would have predicted wouldn’t happen.
*There have been a few cases where the growth didn’t look hyperbolic, but rather like a steady exponential trend that then turns into a singularity. World GDP, by contrast, has what looks like at least three exponential trends in it, such that it is more parsimonious to model it as hyperbolic growth. I think.
I should add though that I haven’t systematically examined these graphs yet, so it’s possible I’m just missing something—e.g. it occurs to me right now that maybe some of these graphs I saw were really logistic functions rather than hyperbolic or exponential-until-you-hit-limits. I should make some more and look at them more carefully.
Yes, thanks! I mostly agree with that assessment,* though as an aside I have a beef with the implication that Bostrom, Yudkowsky, etc. expect discontinuities. That beef is with Paul Christiano, not you. :)
So far the biggest update this has been for me, I think, is that it seems to have shown that it’s quite possible to get an intelligence explosion even without economic feedback loops. Like, even with a fixed compute/money budget—or even with a fixed number of scientists and fixed amount of research funding—we could get singularity. At least in principle. This is weird because in practice I am pretty sure I remember reading that the growth we’ve seen so far can be best explained via an economic feedback loop: Better technology allows for bigger population and economy which allows for more scientists and funding which allows for better technology. So I’m a bit confused, I must say—my model is giving me results I would have predicted wouldn’t happen.
*There have been a few cases where the growth didn’t look hyperbolic, but rather like a steady exponential trend that then turns into a singularity. World GDP, by contrast, has what looks like at least three exponential trends in it, such that it is more parsimonious to model it as hyperbolic growth. I think.
I should add though that I haven’t systematically examined these graphs yet, so it’s possible I’m just missing something—e.g. it occurs to me right now that maybe some of these graphs I saw were really logistic functions rather than hyperbolic or exponential-until-you-hit-limits. I should make some more and look at them more carefully.