I disagree with the author in that I believe the universe is ‘compact’ in the sense that what we know is a near approximation of what is knowable by the method.
For example, I believe our ancestors knew everything they can possibly know without knowing extra background knowledge or special kinds of thinking.
Similarly, I believe we now know nearly everything about the universe (the rules, not specific tools) given use of mathematics and sophisticated logic. And we do a much, much better job than our ancestors at predicting the world. So I am skeptical that superintelligence can exist.
You may say, how can you assume whatever we have today is the final stages of logic? Why can’t this be the very beginning? Again, we nearly explained the universe by now. Second, we have so many interconnected people that it is more than any community in old times by orders of magnitude. So even though logic is harder, I think it is mostly explored.
But this wouldn’t negate AGI, for AGI is different from superintelligence in that we know it is possible (humans, sort of), and also much less powerful than superintelligence.
But one thing we must remember is the hollow nature of economic size. Economy is basically about extracting resources to satisfy our needs. Currently, large economics size is meaningful because it means more resources for each human. And even if a flood of AGI agents increases resource extraction, sums of money in accounts of mere sophisticated algorithms wouldn’t mean anything. At least, not to me.
I disagree with the author in that I believe the universe is ‘compact’ in the sense that what we know is a near approximation of what is knowable by the method.
For example, I believe our ancestors knew everything they can possibly know without knowing extra background knowledge or special kinds of thinking.
Similarly, I believe we now know nearly everything about the universe (the rules, not specific tools) given use of mathematics and sophisticated logic. And we do a much, much better job than our ancestors at predicting the world. So I am skeptical that superintelligence can exist.
You may say, how can you assume whatever we have today is the final stages of logic? Why can’t this be the very beginning? Again, we nearly explained the universe by now. Second, we have so many interconnected people that it is more than any community in old times by orders of magnitude. So even though logic is harder, I think it is mostly explored.
But this wouldn’t negate AGI, for AGI is different from superintelligence in that we know it is possible (humans, sort of), and also much less powerful than superintelligence.
But one thing we must remember is the hollow nature of economic size. Economy is basically about extracting resources to satisfy our needs. Currently, large economics size is meaningful because it means more resources for each human. And even if a flood of AGI agents increases resource extraction, sums of money in accounts of mere sophisticated algorithms wouldn’t mean anything. At least, not to me.