As a result, rather than indefinite and immediate exponential growth, I expect real-world AI growth to follow a series of sigmoidal curves, each eventually plateauing before different types of growth curves take over to increase capabilities based on different input resources (with all of this overlapping).
Hi Andy—how are you gauging the likely relative proportions of AI capability sigmoidal curves relative to the current ceiling of human capability? Unless I’m misreading your position, it seems like you are presuming that the sigmoidal curves will (at least initially) top out at a level that is on the same order as human capabilities. What informs this prior?
Due to the very different nature of our structural limitations (i.e. a brain that’s not too big for a mother’s hips to safely carry and deliver, specific energetic constraints, the not-very-precisely-directed nature of the evolutionary process) vs an AGI’s system’s limitations (which are simply different) it’s totally unclear to me why we should expect the AGI’s plateaus to be found at close-to-human levels.
These curves are due to temporary plateaus, not permanent ones. Moore’s law is an example of a constraint that seems likely to plateau. I’m talking about takeoff speeds, not eventual capabilities with no resource limitations, which I agree would be quite high and I have little idea of how to estimate (there will probably still be some constraints, like within-system communication constraints).
Understood, and agreed, but I’m still left wondering about my question as it pertains to the first sigmoidal curve that shows STEM-capable AGI. Not trying to be nitpicky, just wondering how we should reason about the likelihood that the plateau of that first curve is not already far above the current limit of human capability.
A reason to think so may be something to do with irreducible complexity making things very hard for us at around the same level that it would make them hard for a (first-gen) AGI. But a reason to think the opposite would be that we have line of sight to a bunch of amazing tech already, it’s just a question of allocating the resources to support sufficiently many smart people working out the details.
Another reason to think the opposite is that having a system that’s (in some sense) directly optimized to be intelligent might just have a plateau drawn from a higher-meaned distribution than one that’s optimized for fitness, and develops intelligence as a useful tool in that direction, since the pressure-on-intelligence for that sort of caps out at whatever it takes to dominate your immediate environment.
Hi Andy—how are you gauging the likely relative proportions of AI capability sigmoidal curves relative to the current ceiling of human capability? Unless I’m misreading your position, it seems like you are presuming that the sigmoidal curves will (at least initially) top out at a level that is on the same order as human capabilities. What informs this prior?
Due to the very different nature of our structural limitations (i.e. a brain that’s not too big for a mother’s hips to safely carry and deliver, specific energetic constraints, the not-very-precisely-directed nature of the evolutionary process) vs an AGI’s system’s limitations (which are simply different) it’s totally unclear to me why we should expect the AGI’s plateaus to be found at close-to-human levels.
These curves are due to temporary plateaus, not permanent ones. Moore’s law is an example of a constraint that seems likely to plateau. I’m talking about takeoff speeds, not eventual capabilities with no resource limitations, which I agree would be quite high and I have little idea of how to estimate (there will probably still be some constraints, like within-system communication constraints).
Understood, and agreed, but I’m still left wondering about my question as it pertains to the first sigmoidal curve that shows STEM-capable AGI. Not trying to be nitpicky, just wondering how we should reason about the likelihood that the plateau of that first curve is not already far above the current limit of human capability.
A reason to think so may be something to do with irreducible complexity making things very hard for us at around the same level that it would make them hard for a (first-gen) AGI. But a reason to think the opposite would be that we have line of sight to a bunch of amazing tech already, it’s just a question of allocating the resources to support sufficiently many smart people working out the details.
Another reason to think the opposite is that having a system that’s (in some sense) directly optimized to be intelligent might just have a plateau drawn from a higher-meaned distribution than one that’s optimized for fitness, and develops intelligence as a useful tool in that direction, since the pressure-on-intelligence for that sort of caps out at whatever it takes to dominate your immediate environment.