even if have empirical evidence for the problem being “level n hard” (people have tried up to level n), you;d still do not have empirical evidence for the problem being “level n+1 hard”
This is implicitly assuming that our expectation of how long a problem should take to solve is memoryless. But a breakthrough is much more likely on the 1st day of working on a problem than on the 1000th day. More generally, if problems vary greatly in difficulty, then our failure to solve a given problem provides evidence that it’s one of the harder problems. So a more reasonable prior in this case is something like logarithmic—e.g. it’s equally likely that a problem takes 1-10 days, or 10-100 days, or 100-1000 days, etc, to solve.
A similar model can give rise to the Lindy effect, where the expected lifetime is proportional to the lifetime so far. (In this case it’d be the expected time to solving the problem which would be proportional to the time which the problem has been open.)
This is implicitly assuming that our expectation of how long a problem should take to solve is memoryless. But a breakthrough is much more likely on the 1st day of working on a problem than on the 1000th day. More generally, if problems vary greatly in difficulty, then our failure to solve a given problem provides evidence that it’s one of the harder problems. So a more reasonable prior in this case is something like logarithmic—e.g. it’s equally likely that a problem takes 1-10 days, or 10-100 days, or 100-1000 days, etc, to solve.
A similar model can give rise to the Lindy effect, where the expected lifetime is proportional to the lifetime so far. (In this case it’d be the expected time to solving the problem which would be proportional to the time which the problem has been open.)