The Limits of the Existence Proof Argument for General Intelligence

Link post

If you wish to make an apple pie from scratch, you must first invent the universe.

— Carl Sagan

The existence proof argument for general intelligence, the idea that artificial general intelligence must be possible because human general intelligence exists, would not even be conclusive if by artificial general intelligence we meant a brain grown in a vat — the existence of a thing does not prove that it could have come into existence in any different way — but it’s absolutely absurd if by artificial general intelligence we mean a computer running certain software. The existence of anything does not prove that a computer running certain software could perfectly simulate that thing. This is not even true for many things far simpler then a human brain, such as a double pendulum.

If we took the existence proof argument seriously anyway, we would have no reason to be more concerned about artificial intelligence then we are about human intelligence, we would have no reason to be more concerned about an artificial intelligence explosion then we are about a human intelligence explosion, about a human reaching, with the perfect notes app and the perfect nootropics mix, a level of intelligence that then allows him to become even more intelligent ever faster, turning the entire universe into brain mass in the process.

Concerns about general artificial intelligence are based on the idea that it would be fundamentally different from human intelligence. But the existence of one thing does not prove the possibility of a fundamentally different thing.