Ian, there’s nothing wrong with reductionism.
Overly simplistic reductionism is wrong, e.g., if you divide a computer into individual bits, each of which can be in one of two states, then you can’t explain the operation of the computer in just the states of its bits. However, that reduction omitted an important part, the interconnections of the bits—how each affects the others. When you reduce a computer to individual bits and their immediate relationships with other bits, you can indeed explain the whole computer’s operation, completely. (It just becomes unwieldy to do so.)
“I mean if you list all the actions that it’s parts can do alone, the combined thing can have actions that aren’t in that list.”
What are these “actions that aren’t in that list”? They are still aggregations of interactions that take place at a lower level, but we assign meaning to them. The extra “actions” are in our interpretations of the whole, not in the parts or the whole itself.
“This pattern of belief is very hard to justify from a Bayesian perspective. It is just the same hypothesis in both cases. Even if, in the second case, I announce an experimental method and my intent to actually test it, I have not yet experimented and I have not yet received any observational evidence in favor of the hypothesis.”
Always consider the source.
It is the same hypothesis, but if the person telling the story is unknown to me at the outset, then the credibility of the source rises with test plans, as follows.
After the just-so story: This could be any crank making up stories to sell an idea.
After hearing of a possible test: If this person has taken the time to work out how they would test it, then my confidence is significantly increased. They’ve just set themselves apart from the pure story-tellers.
After looking at the thoroughness of their proposed tests: Confidence rises if they’ve shown a willingness to consider lots of ways that their story and testing could be wrong or misleading. Evidence of familiarity with and adherence to scientific methodology does increase credibility.
After hearing that they are committed to performing their tests: Confidence increases a little more. Not much, but a little.
Sure, these increases in confidence are small—tiny compared to the boost from actually performing even one of the tests—but they are not zero.
If I’m already familiar with the story-teller, and have some prior opinion of their experience and capability, then you’re right: hearing about the tests shouldn’t affect my confidence.