you can always make a proper prior that’s better in every way to a given improper prior
[emphasis mine]
“In every way” is too strong. Some improper priors are derived from the optimization of some criterion, so they are the best in a certain specific way. Also, some improper priors give posterior means that, when treated as estimators, have minimax-optimal frequentist risk. I think you mean something more like “better in every way that ought to matter to someone concerned with rationality”.
Since I don’t think it makes sense to speak of the “surprise” associated with an improper distribution, I think the claim you want to make doesn’t cut to the heart of the matter. There are lots of reasons to object to improper priors, but this isn’t well-formed enough to be one of them.
[emphasis mine]
“In every way” is too strong. Some improper priors are derived from the optimization of some criterion, so they are the best in a certain specific way. Also, some improper priors give posterior means that, when treated as estimators, have minimax-optimal frequentist risk. I think you mean something more like “better in every way that ought to matter to someone concerned with rationality”.
Perhaps “better in every case” would work? That is, less surprising no matter what happens?
Since I don’t think it makes sense to speak of the “surprise” associated with an improper distribution, I think the claim you want to make doesn’t cut to the heart of the matter. There are lots of reasons to object to improper priors, but this isn’t well-formed enough to be one of them.