On the object level, there’s a very plausible mechanism why solving some problems can make things worse
Hmm, I see. But it seems to me that the linked post is overoptimistic that making problems legible will cause people to slow down until they are solved. Already, lots of problems that are “legible” to this community are not “legible” enough to make the labs or governments want to slow down. So working to solve them(by our standards, since apparently other people already consider them solved or unimportant) could still be useful.
Of course you could then say that what we should be doing is trying to make problems legible to this community legible to the wider world, which is pretty much what MIRI is trying to do at the moment. Certainly that seems like a valuable thing to do. But far from guaranteed to succeed. And I think the fact that LW-legibility of takeover isn’t on its own enough to cause the wider world to slow down, should make us less worried that solving one problem to our standards will make the world push ahead more recklessly, since they seemingly aren’t that responsive to what problems are considered solved by our standards.
Hmm, I see. But it seems to me that the linked post is overoptimistic that making problems legible will cause people to slow down until they are solved. Already, lots of problems that are “legible” to this community are not “legible” enough to make the labs or governments want to slow down. So working to solve them(by our standards, since apparently other people already consider them solved or unimportant) could still be useful.
Of course you could then say that what we should be doing is trying to make problems legible to this community legible to the wider world, which is pretty much what MIRI is trying to do at the moment. Certainly that seems like a valuable thing to do. But far from guaranteed to succeed. And I think the fact that LW-legibility of takeover isn’t on its own enough to cause the wider world to slow down, should make us less worried that solving one problem to our standards will make the world push ahead more recklessly, since they seemingly aren’t that responsive to what problems are considered solved by our standards.