Humans are not currently capable of self improvement in the understanding your o. I was talking about the subset of worlds where research talent ense. The “self improvement” section in bookstores doesn’t change the hardware or the operating system, it basically adds more data.
I’m not sure I understand this. Are you claiming is not positive for humans?
In most of the scenarios where the first smarter than human AI, is orders of magnitude faster than a human, I would expect a hard takeoff.
This sounds like “conditioned on a hard takeoff, I expect a hard takeoff”. It’s not exactly saying that, since speed could be different from intelligence, but you need to argue for the premise too: nearly all of the arguments in the linked post could be applied to your premise as well.
In a world where researchers have little idea what they are doing, and are running a new AI every hour hoping to stumble across something that works, the result holds.
In a world where research involves months thinking about maths, then a day writing code, then an hour running it, this result holds.
Agreed on both counts, and again I think the arguments in the linked posts suggest that the premises are not true.
As we went from having no algorithms that could say (tell a cat from a dog) straight to having algorithms superhumanly fast at doing so, there was no algorithm that worked, but took supercomputer hours, this seems like a plausible assumption.
This seems false to me. At what point would you say that we had AI systems that could tell a cat from a dog? I don’t know the history of object recognition, but I would guess that depending on how you operationalize it, I think the answer could be anywhere between the 60s and “we still can’t do it”. (Though it’s also possible that people didn’t care about object recognition until the 21st century, and only did other types of computer vision in the 60s-90s. It’s quite strange that object recognition is an “interesting” task, given how little information you get from it.)
Is the claim that δo/δr for humans goes down over time so that o eventually hits an asymptote? If so, why won’t that apply to AI?
But it seems quite relevant that we haven’t successfully done that yet.
Okay, so my new story for this argument is:
For every task T, there are bottlenecks that limit its performance, which could be compute, data, algorithms, etc.
For the task of “AI research”, compute will not be the bottleneck.
So, once we get human-level performance on “AI research”, we can apply more compute to get exponential recursive self-improvement.
Is that your argument? If so, I think my question would be “why didn’t the bottleneck in point 2 vanish in point 3?” I think the only way this would be true would be if the bottleneck was algorithms, and there was a discontinuous jump in the capability of algorithms. I agree that in that world you would see a hard/fast/discontinuous takeoff, but I don’t see why we should expect that (again, the arguments in the linked posts argue against that premise).