I did overlook the definition of H. Apologies.

The point is that the behavior of H is paradoxical. We can prove that it can’t return true or false without contradiction. But if that’s provable, that also creates a contradiction, since H can prove it to.

More precisely, H will encounter a proof that the question is undecidable. It then runs into the following two if statements:

if check_if_proof_proves_x_halts(proof, x, i)

if check_if_proof_proves_x_doesnt_halt(proof, x, i)

Both return “false”, so H moves into the next iteration of the while loop. H will generate undecidability proofs, but as implemented it will merely discard them and continue searching. Since such proofs do not cause H to halt, and since there are no proofs that the program halts or does not, then H will run forever.

Let’s say I wanted to solve my dating issues. I present the following approaches:

I endeavor to solve the general problem of human sexual attraction, plug myself into the parameters to figure out what I’d be most attracted to, determine the probabilities that individuals I’d be attracted to would also be attracted to me, then devise a strategy for finding someone with maximal compatibility.

I take an iterative approach: I devise a model this afternoon, test it this evening, then analyze the results tomorrow morning and make the necessary adjustments.

Which approach is more rational? Given sufficient time, Approach 1 will yield the optimal solution. Approach 2 has to deal with the problem of local maxima and in the long run is likely to end up worse than Approach 1. An immortal living in an eternal universe would probably say that Approach 1 is vastly superior. Humans, on the other hand, will die well before Approach 1 bears fruit.

While rationality can lead to faster improvement using Approach 2, a rationalist might try Approach 1, whereas a non-rationalist is unlikely to use Approach 1 at all.

Simple amendments to the general problem such as “find the best way to get the best date for next Saturday” will likely lead to solutions making heavy use of deception. If you want to exclude the Dark Arts from the solution space, then that’s going to limit what you can accomplish. The short-term drawbacks of insisting on truth and honesty are well-documented.