I propose that “I don’t know” between fully co-operative rationalists is shorthand for “my knowledge is so weak that I expect you would find negative value in listening to it.” Note that this means whether I say “I don’t know” depends in part on my model of you.
For example, if someone who rarely dabbles in medicine asks me if I think a cure works, and I’ve only skimmed the paper that proposes it, I might well explain how low the prior is and how shaky this sort of research tends to be. If an expert asked me the same question, I’d say “I don’t know” because they already know all that and are asking if I have any unique insight, which I don’t.
Similarly, if someone asks how much a box weighs, and I’m 95% confident it’s between 10 and 50 pounds, I’ll say “I don’t know”, because that range is too wide to be useful for most purposes. But if they follow up with “I’m thinking of shipping it fedex which has a 70 pound maximum”, then I can answer “I’m 95% confident it’s less than 70 pounds.” Though if they also say that if the shipment doesn’t go smoothly the mafia will kill them, my answer is “the scale’s in the bathroom”, because now 95% confidence isn’t good enough.
This does mean that “I don’t know” is a valid answer if my knowledge is so uncompressible that it cannot be transmitted within your patience. I don’t have a good example for this, but I don’t see it as a problem.
I propose that “I don’t know” between fully co-operative rationalists is shorthand for “my knowledge is so weak that I expect you would find negative value in listening to it.” Note that this means whether I say “I don’t know” depends in part on my model of you.
For example, if someone who rarely dabbles in medicine asks me if I think a cure works, and I’ve only skimmed the paper that proposes it, I might well explain how low the prior is and how shaky this sort of research tends to be. If an expert asked me the same question, I’d say “I don’t know” because they already know all that and are asking if I have any unique insight, which I don’t.
Similarly, if someone asks how much a box weighs, and I’m 95% confident it’s between 10 and 50 pounds, I’ll say “I don’t know”, because that range is too wide to be useful for most purposes. But if they follow up with “I’m thinking of shipping it fedex which has a 70 pound maximum”, then I can answer “I’m 95% confident it’s less than 70 pounds.” Though if they also say that if the shipment doesn’t go smoothly the mafia will kill them, my answer is “the scale’s in the bathroom”, because now 95% confidence isn’t good enough.
This does mean that “I don’t know” is a valid answer if my knowledge is so uncompressible that it cannot be transmitted within your patience. I don’t have a good example for this, but I don’t see it as a problem.