There is a huge amount of concepts that we are still unable to describe mathematically. Recursive self-improvement might sound intuitively appealing, but it is nothing that will just happen. Just like friendliness, it takes explicit, mathematically precise definitions to cause an artificial agent to undergo recursive self-improvement.
Machine autocatalysis is already happening. That’s the point of my The Intelligence Explosion Is Happening Now essay. Whatever tech is needed to result in self-improvement is already out there—and the ball is rolling. What happens next is that the man-machine civilisation becomes more machine. That’s the well-known process of automation. The whole process is already self-improving, and it has been since the first living thing.
Self-improvement is not really something where we get to decide whether to build it in.
Machine autocatalysis is already happening. That’s the point of my The Intelligence Explosion Is Happening Now essay. Whatever tech is needed to result in self-improvement is already out there—and the ball is rolling. What happens next is that the man-machine civilisation becomes more machine. That’s the well-known process of automation. The whole process is already self-improving, and it has been since the first living thing.
Self-improvement is not really something where we get to decide whether to build it in.