I don’t think it’s impossible. I have wide uncertainty about timelines, and I certainly think that parts of our systems can get much more efficient. I should have made this more clear in the post. What I’m trying to say is that I am skeptical of a catch-all general efficiency gain that comes from a core insight into rationality, that makes systems much more efficient suddenly.
Imagine a search algorithm that finds local minima, similar to gradient descent, but has faster big-O performance than gradient descent. (For instance, an efficient roughly-n^2 matrix multiplication algorithm would likely yield such a thing, by making true Newton steps tractable on large systems—assuming it played well with sparsity.) That would be a general efficiency gain, and would likely stem from some sudden theoretical breakthrough (e.g. on fast matrix multiplication). And it is exactly the sort of thing which tends to come from a single person/team—the gradual theoretical progress we’ve seen on matrix multiplication is not the kind of breakthrough which makes the whole thing practical; people generally think we’re missing some key idea which will make the problem tractable.
some sudden theoretical breakthrough (e.g. on fast matrix multiplication)
These sorts of ideas seem possible, and I’m not willing to discard them as improbable just yet. I think a way to imagine my argument is that I’m saying, “Hold on, why are we assuming that this is the default scenario? I think we should be skeptical by default.” And so in general counterarguments of the form, “But it could be wrong because of this” aren’t great, because something being possible does not imply that it’s likely.
I don’t think it’s impossible. I have wide uncertainty about timelines, and I certainly think that parts of our systems can get much more efficient. I should have made this more clear in the post. What I’m trying to say is that I am skeptical of a catch-all general efficiency gain that comes from a core insight into rationality, that makes systems much more efficient suddenly.
Imagine a search algorithm that finds local minima, similar to gradient descent, but has faster big-O performance than gradient descent. (For instance, an efficient roughly-n^2 matrix multiplication algorithm would likely yield such a thing, by making true Newton steps tractable on large systems—assuming it played well with sparsity.) That would be a general efficiency gain, and would likely stem from some sudden theoretical breakthrough (e.g. on fast matrix multiplication). And it is exactly the sort of thing which tends to come from a single person/team—the gradual theoretical progress we’ve seen on matrix multiplication is not the kind of breakthrough which makes the whole thing practical; people generally think we’re missing some key idea which will make the problem tractable.
These sorts of ideas seem possible, and I’m not willing to discard them as improbable just yet. I think a way to imagine my argument is that I’m saying, “Hold on, why are we assuming that this is the default scenario? I think we should be skeptical by default.” And so in general counterarguments of the form, “But it could be wrong because of this” aren’t great, because something being possible does not imply that it’s likely.