You should expect that, on average, a test will leave your beliefs unchanged.
Not quite. You do a test because you expect your beliefs to change. A better phrasing is “You should not expect that a test will move your beliefs in any particular direction.” Of course this doesn’t capture the theorem that “prior = expected posterior”, but that is very hard to communicate accurately in English without referring directly to probability theory concepts. At least strive for not having alternate interpretations that are wrong.
I would add There are two kinds of “no evidence”. There’s “no evidence for X” because there’s no evidence either way because X hasn’t been tested, and “no evidence for X” where it’s been tested and all the evidence points to not-X. People often use the first kind of “no evidence” as if had the same force as the second. This is totally obvious under Bayesianism, but not widely understood among the scientifically literate.
See this for another example of confusion between the two kinds of “no evidence”. I summarize:
People think that Cognitive Behavioral Therapy (CBT) is better than psychodynamic/Freudian therapies. This is because CBT has been tested to be better than placebo, but Freudian therapies have not been tested at all; mainly due to historical reasons. Of course, the fact that psychodynamic therapies have not been tested and therefore have no evidence in their favor, isn’t evidence against psychodynamic therapies. They simply have no evidence either way.
And when they went out tested CBT, psychodynamic therapies and placebo, they found that CBT and psychodynamics were about equally better than placebo.
Not quite. You do a test because you expect your beliefs to change. A better phrasing is “You should not expect that a test will move your beliefs in any particular direction.” Of course this doesn’t capture the theorem that “prior = expected posterior”, but that is very hard to communicate accurately in English without referring directly to probability theory concepts. At least strive for not having alternate interpretations that are wrong.
I would add There are two kinds of “no evidence”. There’s “no evidence for X” because there’s no evidence either way because X hasn’t been tested, and “no evidence for X” where it’s been tested and all the evidence points to not-X. People often use the first kind of “no evidence” as if had the same force as the second. This is totally obvious under Bayesianism, but not widely understood among the scientifically literate.
See this for another example of confusion between the two kinds of “no evidence”. I summarize:
People think that Cognitive Behavioral Therapy (CBT) is better than psychodynamic/Freudian therapies. This is because CBT has been tested to be better than placebo, but Freudian therapies have not been tested at all; mainly due to historical reasons. Of course, the fact that psychodynamic therapies have not been tested and therefore have no evidence in their favor, isn’t evidence against psychodynamic therapies. They simply have no evidence either way.
And when they went out tested CBT, psychodynamic therapies and placebo, they found that CBT and psychodynamics were about equally better than placebo.