I think I can get close to saving the world with just one question, though it uses a dirty conditional trick:
If P=NP, please give me an algorithm in P that solves SAT. The degree of the polynomial has to be as small as possible. (Defining a good way to minimize the coefficients for a given degree is trickier, left as an exercise to the reader.)
Otherwise please give me a small and fast algorithm that finds short formal proofs for a list of important theorems from human mathematics. (How to weigh algorithm size, running time, and size of output is left as an exercise to the reader.)
This is also my first reaction, but it has two problems.
First, I think the best case is really just getting a much weaker machine like the one whose existence we are already positing. So maybe it can help you bridge the gap from the first scenario to the second, but I doubt it will be helpful in the second.
Second, I think its likely you would get something horribly, horribly strange out. It gives you proofs which you simply cannot comprehend, that don’t even fit in the language of modern mathematics. Of course analyzing the proofs that come out is an incredibly interesting exercise, but I think I could imagine totally inscrutable 100 page formal proofs of the Riemann Hypothesis which are harder to understand than the Riemann Hypothesis is to prove directly. Human society really doesn’t want proofs of hard theorems; it wants to advance human mathematics. This issue might come up if we develop some unexpectedly powerful automated theorem prover, so I guess it is interesting in its own right (and might have been discussed before).
This is an issue that has already come up to some extent. The initial proof of the Robbins conjecture used automated proving systems for a large amount of it, and they made no intuitive sense. It took about 2 years for someone to grok the proof well enough to present a version that made sense to people.
So maybe it can help you bridge the gap from the first scenario to the second, but I doubt it will be helpful in the second.
Hm, I thought it was already obvious what to do in the second scenario, and the only challenge was getting from the first to the second. See my reply to Nesov.
This is one of the issues where LW potentially disagrees with the rest of humanity, and I think the LW position (or at least the position you articulate) is actually wrong. I see many object-level ways to help the world once we have a magical optimizer: solve protein folding, model plasma containment, etc. And these are just the opportunities we can take advantage of with existing tech, but the optimizer can also help design new tech.
And these are just the opportunities we can take advantage of with existing tech, but the optimizer can also help design new tech.
We can do lots of useful things, sure (this is not a point where we disagree), but they don’t add up towards “saving the world”. These are just short-term benefits. Technological progress makes it easier to screw stuff up irrecoverably, advanced tech is the enemy. One shouldn’t generally advance the tech if distant end-of-the-world is considered important as compared to immediate benefits (this value judgment can well be a real point of disagreement).
Modeling physical systems is already hard. I don’t think we could yet write down the dynamics of the physical systems well enough (or rather, we don’t understand what the most important characteristics are) to come up with a precise formulation of the major problems in synthetic biology or nanotechnology. I certainly concede that an optimizer would be helpful in solving many subproblems, and would considerably increase the speed of new developments in pretty much every field. I don’t think it solves many problems on its own though.
But even if you could solve narrow existing technological problems or develop new technologies at a steady pace, it seems like you should be able to do more. Suppose the box can do in a minute what takes existing humans a million years. Then our only upper bound on our capabilities using the box is whatever we expect of a million years of progress at the current pace. I don’t know about you, but I expect pretty much everything.
I think I can get close to saving the world with just one question, though it uses a dirty conditional trick:
If P=NP, please give me an algorithm in P that solves SAT. The degree of the polynomial has to be as small as possible. (Defining a good way to minimize the coefficients for a given degree is trickier, left as an exercise to the reader.)
Otherwise please give me a small and fast algorithm that finds short formal proofs for a list of important theorems from human mathematics. (How to weigh algorithm size, running time, and size of output is left as an exercise to the reader.)
This is also my first reaction, but it has two problems.
First, I think the best case is really just getting a much weaker machine like the one whose existence we are already positing. So maybe it can help you bridge the gap from the first scenario to the second, but I doubt it will be helpful in the second.
Second, I think its likely you would get something horribly, horribly strange out. It gives you proofs which you simply cannot comprehend, that don’t even fit in the language of modern mathematics. Of course analyzing the proofs that come out is an incredibly interesting exercise, but I think I could imagine totally inscrutable 100 page formal proofs of the Riemann Hypothesis which are harder to understand than the Riemann Hypothesis is to prove directly. Human society really doesn’t want proofs of hard theorems; it wants to advance human mathematics. This issue might come up if we develop some unexpectedly powerful automated theorem prover, so I guess it is interesting in its own right (and might have been discussed before).
This is an issue that has already come up to some extent. The initial proof of the Robbins conjecture used automated proving systems for a large amount of it, and they made no intuitive sense. It took about 2 years for someone to grok the proof well enough to present a version that made sense to people.
Hm, I thought it was already obvious what to do in the second scenario, and the only challenge was getting from the first to the second. See my reply to Nesov.
Even having a magical computational-efficiency-optimizer won’t currently help with saving the world. Can easily help with destroying it though.
This is one of the issues where LW potentially disagrees with the rest of humanity, and I think the LW position (or at least the position you articulate) is actually wrong. I see many object-level ways to help the world once we have a magical optimizer: solve protein folding, model plasma containment, etc. And these are just the opportunities we can take advantage of with existing tech, but the optimizer can also help design new tech.
We can do lots of useful things, sure (this is not a point where we disagree), but they don’t add up towards “saving the world”. These are just short-term benefits. Technological progress makes it easier to screw stuff up irrecoverably, advanced tech is the enemy. One shouldn’t generally advance the tech if distant end-of-the-world is considered important as compared to immediate benefits (this value judgment can well be a real point of disagreement).
I agree with Nesov’s response, and would be interested to know if you’ve changed your mind since writing this comment.
Modeling physical systems is already hard. I don’t think we could yet write down the dynamics of the physical systems well enough (or rather, we don’t understand what the most important characteristics are) to come up with a precise formulation of the major problems in synthetic biology or nanotechnology. I certainly concede that an optimizer would be helpful in solving many subproblems, and would considerably increase the speed of new developments in pretty much every field. I don’t think it solves many problems on its own though.
But even if you could solve narrow existing technological problems or develop new technologies at a steady pace, it seems like you should be able to do more. Suppose the box can do in a minute what takes existing humans a million years. Then our only upper bound on our capabilities using the box is whatever we expect of a million years of progress at the current pace. I don’t know about you, but I expect pretty much everything.