Is it possible that humans might create blight power AI, sure. Is it possible that a monkey banging away on a keyboard might create the complete works of Shakespeare, sure. I’m not going to hold my breath though.
If groups of humans do manage to cobble together an AGI out of half baked theories and random trial and error, it is likely to have as much hope of recursively self-improving easily as a singular human performing neurosurgery on themselves. Even given the tools to alter neural connections and weightings without damage, I don’t see much hope of quick improvement.
Power level intelligence requires power level optimisation power to create out of nothing. If you can create a power level intelligence, that optimises in the same way that you would wish to, then by the definition you have given for optimisation power the creator of that intelligence must have it.
Developing something that can become a power level AI first would be like accidentally creating a space ship when trying to fly for the first time. Trying to hit a infinitesimal target in optimisation space, when you don’t even know if you are in the right ball park.
One of the main benefits I see from real AI, is the intellectual shockwave that will hit humanity when we can demonstrate that intellect is naturalistic. A deep understanding of what we are is necessary for further growth of humanity.
Is it possible that humans might create blight power AI, sure. Is it possible that a monkey banging away on a keyboard might create the complete works of Shakespeare, sure. I’m not going to hold my breath though.
If groups of humans do manage to cobble together an AGI out of half baked theories and random trial and error, it is likely to have as much hope of recursively self-improving easily as a singular human performing neurosurgery on themselves. Even given the tools to alter neural connections and weightings without damage, I don’t see much hope of quick improvement.
Power level intelligence requires power level optimisation power to create out of nothing. If you can create a power level intelligence, that optimises in the same way that you would wish to, then by the definition you have given for optimisation power the creator of that intelligence must have it.
Developing something that can become a power level AI first would be like accidentally creating a space ship when trying to fly for the first time. Trying to hit a infinitesimal target in optimisation space, when you don’t even know if you are in the right ball park.
One of the main benefits I see from real AI, is the intellectual shockwave that will hit humanity when we can demonstrate that intellect is naturalistic. A deep understanding of what we are is necessary for further growth of humanity.