One way to train this: in my number theory class, there was a type of problem called a PODASIP. This stood for Prove Or Disprove And Salvage If Possible. The instructor would give us a theorem to prove, without telling us if it was true or false. If it was true, we were to prove it. If it was false, then we had to disprove it and then come up with the “most general” theorem similar to it (e.g. prove it for Zp after coming up with a counterexample in Zm).
This trained us to be on the lookout for problems with the theorem, but then seeing the “least convenient possible world” in which it was true.
“Act as if” might work.
For example, I act as if people are nicer than they are (because it gets me better outcomes than other possible strategies I’ve tried).
This also has the benefit of clearly separating action (what we can do) from information (what we know) and preferences (what we want).