Richard Hamming had this to say about important problems, in his talk “You and Your Research”:
Let me warn you, “important problem” must be phrased carefully. The three outstanding problems in physics, in a certain sense, were never worked on while I was at Bell Labs. By important I mean guaranteed a Nobel Prize and any sum of money you want to mention. We didn’t work on (1) time travel, (2) teleportation, and (3) antigravity. They are not important problems because we do not have an attack. It’s not the consequence that makes a problem important, it is that you have a reasonable attack.
One reasonable attack makes the problem approachable. If there are multiple reasonable attacks, at least one succeeding becomes more likely and further they can exchange information about the problem making each attempt more likely on its own. If we switch to considering thoroughly understood problems, we usually have multiple good solutions for them (like multiple proofs in mathematics, or detection from different kinds of experimental apparatus in science).
So if I am going to rank open problems by the likelihood they will be solved, my prior is a list ordered by the number of ways we know of to attack each problem. Without any other information, a problem with two reasonable attacks is twice as likely to be solved as a problem with only one.
Then we could consider updating the weights of different kinds of attack. For example, if one requires very expensive equipment, or very rare expertise, I might adjust it down. On the other hand, if there are two different attacks but the relationship between those two approaches is otherwise very well understood, then we might not treat them as independent anymore and factor in the ease of sharing information between them but also that they will probably succeed or fail together.
We can also consider the problem itself, but I feel like looking at the reference classes for a problem largely boils down to a way to search for reasonable attacks, where any attack which worked for a problem in the reference class is considered a candidate for the problem at hand. But as I think of it, I’m not sure it is common to do a systematic evaluation in this way, so highlighting it as a specific method for finding attacks seems worthwhile.
Multiple angles of attack
Richard Hamming had this to say about important problems, in his talk “You and Your Research”:
One reasonable attack makes the problem approachable. If there are multiple reasonable attacks, at least one succeeding becomes more likely and further they can exchange information about the problem making each attempt more likely on its own. If we switch to considering thoroughly understood problems, we usually have multiple good solutions for them (like multiple proofs in mathematics, or detection from different kinds of experimental apparatus in science).
So if I am going to rank open problems by the likelihood they will be solved, my prior is a list ordered by the number of ways we know of to attack each problem. Without any other information, a problem with two reasonable attacks is twice as likely to be solved as a problem with only one.
Then we could consider updating the weights of different kinds of attack. For example, if one requires very expensive equipment, or very rare expertise, I might adjust it down. On the other hand, if there are two different attacks but the relationship between those two approaches is otherwise very well understood, then we might not treat them as independent anymore and factor in the ease of sharing information between them but also that they will probably succeed or fail together.
We can also consider the problem itself, but I feel like looking at the reference classes for a problem largely boils down to a way to search for reasonable attacks, where any attack which worked for a problem in the reference class is considered a candidate for the problem at hand. But as I think of it, I’m not sure it is common to do a systematic evaluation in this way, so highlighting it as a specific method for finding attacks seems worthwhile.