As far as I understand, your point (2) is too weak. The claim is not that the AI will merely be smarter than us humans by some margin; instead, the claim is that (2a) the AI will become so smart that it will become a different category of being, thus ushering in a Singularity. Some people go so far as to claim that the AI’s intelligence will be effectively unbounded.
I personally do not doubt that (1) is true (after all, humans are recursively self-improving entities, so we know it’s possible), and that your weaker form of (2) is true (some humans are vastly smarter than average, so again, we know it’s possible), but I am not convinced that (2a) is true.
As far as I understand, your point (2) is too weak. The claim is not that the AI will merely be smarter than us humans by some margin; instead, the claim is that (2a) the AI will become so smart that it will become a different category of being, thus ushering in a Singularity. Some people go so far as to claim that the AI’s intelligence will be effectively unbounded.
I personally do not doubt that (1) is true (after all, humans are recursively self-improving entities, so we know it’s possible), and that your weaker form of (2) is true (some humans are vastly smarter than average, so again, we know it’s possible), but I am not convinced that (2a) is true.