Some questions/answers are hard to think of, not because we are missing knowledge or technology but because they are hard to think of.
I guess this is basically case (1), but I think “vanishingly small” is a bad description in most fields. For example, a significant fraction of advances in theoretical fields are of this form (the exception being for rapidly developing fields). I agree that if you think you have found a proof that P!=NP you should stop and reflect on how many people have thought for a very long time about it, but in general you should not be shocked that you have thought of a question or answer that eluded other researchers, potentially for a rather long time (since there are very many questions and not that many researchers equipped to ask or answer most of them).
I guess you are claiming that the situation is fundamentally different for experimentalists, but I would still be rather shocked if (1) was really negligible. At best there may be some fields where (1) is particularly uncommon (and for people who are still deciding what they want to do with their lives, it seems like this suggests a lot of interesting data which are really hard to extract from the normal discourse about possible fields of study).
“Vanishingly small” was a bit incendiary, and it will depend on the field. This is why theoretical physics, where (1) is common, seems so scary and ego-driven, it’s all about how smart you are relative to your peers.
But in general I think it’s correct to say that the odds your idea is so special really are low. The equal odds rule says that the average publication of any particular scientist does not have any statistically different chance of having more of an impact (i.e., more citations) than any other scientist’s average publication. Most papers are lost to the ether, and published papers are really just the tip of the iceburg of most ideas.
Some questions/answers are hard to think of, not because we are missing knowledge or technology but because they are hard to think of.
I guess this is basically case (1), but I think “vanishingly small” is a bad description in most fields. For example, a significant fraction of advances in theoretical fields are of this form (the exception being for rapidly developing fields). I agree that if you think you have found a proof that P!=NP you should stop and reflect on how many people have thought for a very long time about it, but in general you should not be shocked that you have thought of a question or answer that eluded other researchers, potentially for a rather long time (since there are very many questions and not that many researchers equipped to ask or answer most of them).
I guess you are claiming that the situation is fundamentally different for experimentalists, but I would still be rather shocked if (1) was really negligible. At best there may be some fields where (1) is particularly uncommon (and for people who are still deciding what they want to do with their lives, it seems like this suggests a lot of interesting data which are really hard to extract from the normal discourse about possible fields of study).
“Vanishingly small” was a bit incendiary, and it will depend on the field. This is why theoretical physics, where (1) is common, seems so scary and ego-driven, it’s all about how smart you are relative to your peers.
But in general I think it’s correct to say that the odds your idea is so special really are low. The equal odds rule says that the average publication of any particular scientist does not have any statistically different chance of having more of an impact (i.e., more citations) than any other scientist’s average publication. Most papers are lost to the ether, and published papers are really just the tip of the iceburg of most ideas.