Case 1) Information that I know. I have enough information to come to a particular conclusion with reasonable confidence. If some other people might not have reached the conclusion, and its useful or interesting, then I might share it. So I don’t share things that everyone knows, or things that no one cares about.
Case 2) The information is available, I have not done research and formed a conclusion. This covers cases where I don’t know whats going on, because I can’t be bothered to find out. I don’t know who won sportsball. What use is there in telling everyone my null prior.
Case 3) The information is not readily available. If I think a question is important, and I don’t know the answer already, then the answer is hard to get. Maybe no-one knows the answer, maybe the answer is all in jargon that I don’t understand. For example “Do aliens exist?”. Sometimes a little evidence is available, and speculative conclusions can be drawn. But is sharing some faint wisps of evidence, and describing a posterior that’s barely been updated saying wrong things?
On a societal level, if you set a really high bar for reliability, all you get is the vacuously true. Set too low a bar, and almost all the conclusions will be false. Don’t just have a pile of hypotheses that are at least n% likely to be true, for some fixed n. Keep your hypothesis sorted by likelihood. A place for near certainties. A place for conclusions that are worth considering for the 1% chance they are correct.
Of course, in a large answer space, where the amount of evidence available and the amount required are large and varying, the chance that both will be within a few bits of each other is small. Suppose the correct hypothesis takes some random number of bits between 1 and 10,000 to locate. And suppose the evidence available is also randomly spread between 1 and 10,000. The chance of the two being within 10 bits of each other is about 1⁄500.
This means that 499 times out of 500, you assign the correct hypothesis a chance of less than 0.1% or more than 99.9%. Uncertain conclusions are rare.
Lets consider the different cases seperately.
Case 1) Information that I know. I have enough information to come to a particular conclusion with reasonable confidence. If some other people might not have reached the conclusion, and its useful or interesting, then I might share it. So I don’t share things that everyone knows, or things that no one cares about.
Case 2) The information is available, I have not done research and formed a conclusion. This covers cases where I don’t know whats going on, because I can’t be bothered to find out. I don’t know who won sportsball. What use is there in telling everyone my null prior.
Case 3) The information is not readily available. If I think a question is important, and I don’t know the answer already, then the answer is hard to get. Maybe no-one knows the answer, maybe the answer is all in jargon that I don’t understand. For example “Do aliens exist?”. Sometimes a little evidence is available, and speculative conclusions can be drawn. But is sharing some faint wisps of evidence, and describing a posterior that’s barely been updated saying wrong things?
On a societal level, if you set a really high bar for reliability, all you get is the vacuously true. Set too low a bar, and almost all the conclusions will be false. Don’t just have a pile of hypotheses that are at least n% likely to be true, for some fixed n. Keep your hypothesis sorted by likelihood. A place for near certainties. A place for conclusions that are worth considering for the 1% chance they are correct.
Of course, in a large answer space, where the amount of evidence available and the amount required are large and varying, the chance that both will be within a few bits of each other is small. Suppose the correct hypothesis takes some random number of bits between 1 and 10,000 to locate. And suppose the evidence available is also randomly spread between 1 and 10,000. The chance of the two being within 10 bits of each other is about 1⁄500.
This means that 499 times out of 500, you assign the correct hypothesis a chance of less than 0.1% or more than 99.9%. Uncertain conclusions are rare.