Wei, essentially the black box has to be such that it seems more likely for it to be a halting oracle than something only slightly smarter than you are—since when it says “doesn’t halt”, and you don’t know if it halts or not, it’s not like you can easily check.
Suppose, though, that the black box has the apparent qualities of a closed timelike curve in a universe that permits CTCs to be constructed with perfection (even if our own universe permitted CTCs they would break down well before up-arrow time). Then it would seem pretty plausible, after constructing the CTC and verifying its apparent performance on the cases you knew how to test, that the CTC wasn’t a trickster SI in disguise.
But -
in this case, you can just build a CTC inside yourself, as a closed system, and examine the results with your eyes shut. Your “complexity” won’t change, it’ll just have to be defined outside the Turing formalism.
So once again I say: it is really hard to improve your math abilities with eyes open in a way that you couldn’t theoretically do with eyes closed.
Wei, essentially the black box has to be such that it seems more likely for it to be a halting oracle than something only slightly smarter than you are—since when it says “doesn’t halt”, and you don’t know if it halts or not, it’s not like you can easily check.
Suppose, though, that the black box has the apparent qualities of a closed timelike curve in a universe that permits CTCs to be constructed with perfection (even if our own universe permitted CTCs they would break down well before up-arrow time). Then it would seem pretty plausible, after constructing the CTC and verifying its apparent performance on the cases you knew how to test, that the CTC wasn’t a trickster SI in disguise.
But -
in this case, you can just build a CTC inside yourself, as a closed system, and examine the results with your eyes shut. Your “complexity” won’t change, it’ll just have to be defined outside the Turing formalism.
So once again I say: it is really hard to improve your math abilities with eyes open in a way that you couldn’t theoretically do with eyes closed.