You are missing a train of argument which trumps all of these lines of criticism: the intelligence explosion is already upon us. Creating a modern microprocessor chip is a stupendously complex computational task that is far far beyond the capabilities of any number of un-amplified humans, no matter how intelligent. There was a time when chips were simple enough that a single human could do all of this intellectual work, but we are already decades past that point.
Today new chips are created by modern day super-intelligences (corporations) which in turn are composed of a mix of a variety of specialized meat brains running all kinds of natural language software and all kinds of computers running a complex ecosystem of programs in machine languages. Over time more and more of the total computational work is shifting from the meat brains to the machines. For a more concrete example of just one type of specialized machine that is a node in this overall meta-system, look at the big expensive emulation engines that are used to speed up final testing.
There is a trend to over-emphasis human’s computational/intellectual contributions. When machines get better at particular sub-tasks or domains such as chess, we simply exclude them from our broad notions of ‘intelligence’. Thus we tend to define intelligence at whatever humans are currently best at.
From that perspective I can predict a future intelligence extinction event as eventually all of the tasks move over to the machine substrate and eventually there are no tasks left favoring humans, and we are forced to conclude that intelligence no longer exists.
More seriously, the intelligence explosion is already well under way, we just aren’t seeing it, for the most part. It’s too big and abstract, too far from the simpler more visible physical manifestations we would expect.
That being said, there is certainly a further potential phase transition ahead when machine understanding of natural language matches then exceeds human level, when all of the existing human knowledge moves onto machines and they can run existing natural language ‘software’. I find it likely this will lead to a further stage in the explosion, perhaps a transition to another S-curve of growth and dislocation, but such a transition would not fundamentally change either the computational problems important for progress (such as creating new chips), or the nature of the solutions.
Computers already do a huge chunk of the important work, thus completely taking humans out of the loop can only lead to limited additional gains in these fields where computers are already important (research, engineering).
You are missing a train of argument which trumps all of these lines of criticism: the intelligence explosion is already upon us. Creating a modern microprocessor chip is a stupendously complex computational task that is far far beyond the capabilities of any number of un-amplified humans, no matter how intelligent. There was a time when chips were simple enough that a single human could do all of this intellectual work, but we are already decades past that point.
Today new chips are created by modern day super-intelligences (corporations) which in turn are composed of a mix of a variety of specialized meat brains running all kinds of natural language software and all kinds of computers running a complex ecosystem of programs in machine languages. Over time more and more of the total computational work is shifting from the meat brains to the machines. For a more concrete example of just one type of specialized machine that is a node in this overall meta-system, look at the big expensive emulation engines that are used to speed up final testing.
There is a trend to over-emphasis human’s computational/intellectual contributions. When machines get better at particular sub-tasks or domains such as chess, we simply exclude them from our broad notions of ‘intelligence’. Thus we tend to define intelligence at whatever humans are currently best at.
From that perspective I can predict a future intelligence extinction event as eventually all of the tasks move over to the machine substrate and eventually there are no tasks left favoring humans, and we are forced to conclude that intelligence no longer exists.
More seriously, the intelligence explosion is already well under way, we just aren’t seeing it, for the most part. It’s too big and abstract, too far from the simpler more visible physical manifestations we would expect.
That being said, there is certainly a further potential phase transition ahead when machine understanding of natural language matches then exceeds human level, when all of the existing human knowledge moves onto machines and they can run existing natural language ‘software’. I find it likely this will lead to a further stage in the explosion, perhaps a transition to another S-curve of growth and dislocation, but such a transition would not fundamentally change either the computational problems important for progress (such as creating new chips), or the nature of the solutions.
Computers already do a huge chunk of the important work, thus completely taking humans out of the loop can only lead to limited additional gains in these fields where computers are already important (research, engineering).