All (Non-Trivial) Decisions Are Undecidable

A core tenet of rationalism is that for any given set of known information, a span of possible decisions, and some utility function, there exists a decision-making algorithm that will return the optimal outcome. A short argument, given below, will demonstrate (given relatively minimal assumptions) that finding any such algorithm is an undecidable problem.

We will take as given three axioms:

Running any decision-making algorithm requires a non-negligible amount of time, and incurs a non-negligible search cost over time.
Any above algorithm can be run by a Turing machine^[1].
During decision-making, there always exists some potentially relevant information not known to the decider^[2].

Therefore, any proposed optimal decision-making algorithm, henceforth referred to as D, must also encode an algorithm H, which determines the ideal halting point^[3].
Thus, by the same logic that underlies the halting problem (and axiom 3), there exists for every D a possible state of affairs D’ which contains an ideal halting point H’, such that when D is run, H does not halt D at the ideal halting point^[4]. This, being the definition of undecidability, concludes the argument.

^
Or a nondeterministic Turing machine; according to Wikipedia, they differ only in time complexity.
^
If there were not, one would be locally omniscient, therefore trivializing decision-making.
^
If there were no halting point, the algorithm would not terminate, guaranteeing unbounded search cost (a strictly suboptimal outcome).
^
In common parlance, the possibility that H is not the optimal halting point is referred to as ‘doubt’, and the factors that potentially defy the optimality of D are referred to as ‘unknowns’.