I still think what you’re saying is contradictory. We’re using “rationality” to mean “maximizing expected utility”, correct? If we are aware that certain classes of attempts to do so will be punished, then we’re aware that they will not in fact maximize our expected utility, so by definition such attempts aren’t rational.
It seems like you’re picking and choosing which counterfactuals “count” and which ones don’t. How does punishment differ from any other constraint? If I inhabited a universe in which I had an infinite amount of time and space with which to compute my decisions, I’d implement AIXI and call it good. The universe I actually inhabit requires me to sacrifice that particular form of optimality, but that doesn’t mean it’s irrational to make theoretically sub-optimal decisions.
I still think what you’re saying is contradictory. We’re using “rationality” to mean “maximizing expected utility”, correct? If we are aware that certain classes of attempts to do so will be punished, then we’re aware that they will not in fact maximize our expected utility, so by definition such attempts aren’t rational.
It seems like you’re picking and choosing which counterfactuals “count” and which ones don’t. How does punishment differ from any other constraint? If I inhabited a universe in which I had an infinite amount of time and space with which to compute my decisions, I’d implement AIXI and call it good. The universe I actually inhabit requires me to sacrifice that particular form of optimality, but that doesn’t mean it’s irrational to make theoretically sub-optimal decisions.