I think a lot of this comes down to the question of whether there is something about the “substrate” of the real physical world, and not just the calculations happening in it, that contributes to actual conscious experience and various other features of the true intelligence you describe.
There is a real possibility that such a substrate is inaccessible through normal scientific probing, just like a virtual-machine running on a host bare-metal cannot access the bare-metal unless the host pipes it through.
I know scientist hate such ideas of some horizon behind which we cannot see, or where information is lost/hidden, so this whole train of thought tends to be ignored.
I think a lot of this comes down to the question of whether there is something about the “substrate” of the real physical world, and not just the calculations happening in it, that contributes to actual conscious experience and various other features of the true intelligence you describe.
There is a real possibility that such a substrate is inaccessible through normal scientific probing, just like a virtual-machine running on a host bare-metal cannot access the bare-metal unless the host pipes it through.
I know scientist hate such ideas of some horizon behind which we cannot see, or where information is lost/hidden, so this whole train of thought tends to be ignored.