Yet if we extend the “+1 complexity” argument, we eventually reach a boundary where no human, however smart, could understand it. In principle nature could produce a human with the specific mutation necessary to apprehend it, which pushes the human cognitive horizon by some amount without actually eliminating it.
To the extent that AI can be scaled unlike the human brain, it might be able to form conceptual primitives which are so far outside the human cognitive horizon that biology is unlikely to produce a human intelligent enough to apprehend them on any reasonable timescale.
Yet if we extend the “+1 complexity” argument, we eventually reach a boundary where no human, however smart, could understand it. In principle nature could produce a human with the specific mutation necessary to apprehend it, which pushes the human cognitive horizon by some amount without actually eliminating it.
To the extent that AI can be scaled unlike the human brain, it might be able to form conceptual primitives which are so far outside the human cognitive horizon that biology is unlikely to produce a human intelligent enough to apprehend them on any reasonable timescale.