If you took the original post to mean that weak evidence isn’t common, I’d contend you took the wrong lesson. Encountering strong evidence can be a common occurrence while still being much less common than encountering weak evidence.
You are constantly bombarded by evidence all the time, so something can be true of only a tiny fraction of all evidence you encounter and still be something you experience many times per day.
Also, each observation is simultaneously evidence of many different things. When someone asked if you were evil and you said “no”, that was weak evidence against you being evil, but also fairly strong evidence that you speak English. If I put my hand out the window and it gets wet, that’s pretty strong evidence that it’s raining outside my window, but also weaker evidence that it’s raining on the other side of town.
I think you’re right to point out that just because strong evidence is common in general doesn’t necessarily mean that strong evidence about some specific question you are interested in will be common. There are definitely questions for which it is hard to find strong evidence.
But I don’t especially trust your generalization of when strong evidence should be expected, and I think some of your examples are confused.
In asking how hard it is to get evidence that you can beat the stock market, I think you are misleadingly combining the questions “how likely is it that you can beat the market?” and “in worlds where you can, how hard is it to get evidence of that?” in order to imply that the evidence is hard to get, when I think most of the intuition that you are unlikely to see such evidence is coming from the “it’s unlikely to be true” side. (Also, it is actually not true that getting >50% of trades right means you will be profitable, because the size of the gains or losses matters.)
And asking someone “are you overconfident?” may not give you very much evidence about whether they are overconfident or not, but that’s probably far from the best strategy for gathering evidence on that question.
If you took the original post to mean that weak evidence isn’t common, I’d contend you took the wrong lesson. Encountering strong evidence can be a common occurrence while still being much less common than encountering weak evidence.
You are constantly bombarded by evidence all the time, so something can be true of only a tiny fraction of all evidence you encounter and still be something you experience many times per day.
Also, each observation is simultaneously evidence of many different things. When someone asked if you were evil and you said “no”, that was weak evidence against you being evil, but also fairly strong evidence that you speak English. If I put my hand out the window and it gets wet, that’s pretty strong evidence that it’s raining outside my window, but also weaker evidence that it’s raining on the other side of town.
I think you’re right to point out that just because strong evidence is common in general doesn’t necessarily mean that strong evidence about some specific question you are interested in will be common. There are definitely questions for which it is hard to find strong evidence.
But I don’t especially trust your generalization of when strong evidence should be expected, and I think some of your examples are confused.
In asking how hard it is to get evidence that you can beat the stock market, I think you are misleadingly combining the questions “how likely is it that you can beat the market?” and “in worlds where you can, how hard is it to get evidence of that?” in order to imply that the evidence is hard to get, when I think most of the intuition that you are unlikely to see such evidence is coming from the “it’s unlikely to be true” side. (Also, it is actually not true that getting >50% of trades right means you will be profitable, because the size of the gains or losses matters.)
And asking someone “are you overconfident?” may not give you very much evidence about whether they are overconfident or not, but that’s probably far from the best strategy for gathering evidence on that question.