I am pretty sure the takeoff will require repeatedly doubling and quadrupling hardware, not just autorewriting software.
Do you think that progress in AI is limited primarily by hardware? If hardware is the limiting factor, then you should think AI soon relatively plausible. If software is the limiting factor (the majority view, and the reason most AI folk reject claims such as those of Moravec), such that we won’t get AI until well beyond the minimum computational requirements, then either early AIs should be able to run fast or with numerous copies cheaply, or there will be a lot of room to reduce bloated hardware demands through software improvements.
Thinking that AI will take a long time (during which hardware will advance mightily towards physical limits) but also be sharply and stably hardware-limited when created is a hard view to defend.
I am imagining that it will work something like the human brain (but not by ‘scan and emulate’). We need to create hardware modules comparable to neurons, we need to have some kind of geometric organization which permits individual hardware modules to establish physical connections to a handful of nearby modules, and we need a ‘program’ (corresponding to human embryonic development) which establishes a few starting connections, and finally we need a training period (like training a neural net, and comparable to what the human brain experiences from the first neural activity in the womb through graduate school) which adds many more physical connections. I’m not sure whether to call these connections hardware or software. Actually, they are a hybrid of both—like PLAs (yeah, I way out of date on technology).
So I’m imagining a lot of theoretical work needed to come up with a good ‘neuron’ design (probably several dozen different kinds of neurons), more theoretical work to come up with a good ‘program’ to correspond to the embryonic interconnect, and someone willing to pay for lots and lots of neurons.
So, yeah, I’m thinking that the program will be relatively simple (equivalent to a few million lines of code), but it will take us a long time to find it. Not the 500 million years that it took evolution to come up with that program—apparently 500 million years after it had already invented the neuron. But for human designers, at least a few decades to find and write the program. I hope this explanation helps to make my position seem less weird.
Do you think that progress in AI is limited primarily by hardware? If hardware is the limiting factor, then you should think AI soon relatively plausible. If software is the limiting factor (the majority view, and the reason most AI folk reject claims such as those of Moravec), such that we won’t get AI until well beyond the minimum computational requirements, then either early AIs should be able to run fast or with numerous copies cheaply, or there will be a lot of room to reduce bloated hardware demands through software improvements.
Thinking that AI will take a long time (during which hardware will advance mightily towards physical limits) but also be sharply and stably hardware-limited when created is a hard view to defend.
I am imagining that it will work something like the human brain (but not by ‘scan and emulate’). We need to create hardware modules comparable to neurons, we need to have some kind of geometric organization which permits individual hardware modules to establish physical connections to a handful of nearby modules, and we need a ‘program’ (corresponding to human embryonic development) which establishes a few starting connections, and finally we need a training period (like training a neural net, and comparable to what the human brain experiences from the first neural activity in the womb through graduate school) which adds many more physical connections. I’m not sure whether to call these connections hardware or software. Actually, they are a hybrid of both—like PLAs (yeah, I way out of date on technology).
So I’m imagining a lot of theoretical work needed to come up with a good ‘neuron’ design (probably several dozen different kinds of neurons), more theoretical work to come up with a good ‘program’ to correspond to the embryonic interconnect, and someone willing to pay for lots and lots of neurons.
So, yeah, I’m thinking that the program will be relatively simple (equivalent to a few million lines of code), but it will take us a long time to find it. Not the 500 million years that it took evolution to come up with that program—apparently 500 million years after it had already invented the neuron. But for human designers, at least a few decades to find and write the program. I hope this explanation helps to make my position seem less weird.