If you do not choose robustly generalizable subproblems and find robustly generalizable solutions to them, then most likely, your contribution will not be small; it will be completely worthless.
Though it also feels worth noting that while individually unsatisfying, it’s not necessarily collectively wrong for a lot of people to work on subproblems that will turn out to be completely worthless. If one of them turns out to unexpectedly solve the main bottleneck after all, or if several of them do end up making small progress that lets someone else solve the main bottleneck, all of the “wasted” work may in fact have bought us the useful bits that you couldn’t have identified in advance.
Often when I’ve had a hypothesis about something that interests me, I’ve been happy that there has been *so much* scientific research done on various topics, many of them seemingly insignificant. While most of it is of little interest to me, the fact that there’s so much of it means that there’s often some prior work on topics that do interest me. And vice versa—much of the work that I find uninteresting, will probably be a useful puzzle piece in someone else’s research. (Even if a lot of it is also genuinely useless for everyone, but that’s the price we pay for getting some of the more useful bits.)
That sort of reasoning makes sense insofar as it’s hard to predict which small pieces will be useful. And while that is hard to some extent, it is not full we-just-have-no-idea-so-use-a-maxent-prior hard. There is plenty of work (including lots of research which people sink their lives into today) which will predictably-in-advance be worthless. And robust generalizability is the main test I know of for that purpose.
Applying this to your own argument:
Often when I’ve had a hypothesis about something that interests me, I’ve been happy that there has been *so much* scientific research done on various topics, many of them seemingly insignificant. While most of it is of little interest to me, the fact that there’s so much of it means that there’s often some prior work on topics that do interest me.
It will predictably and systematically be the robustly generalizable things which are relevant to other people in unexpected ways.
Without necessarily accusing Kaj specifically, this general type of argument feels motivated to me. It feels like willful ignorance, like sticking one’s head in the sand and ignoring the available information, because one wants to believe that All Research is Valuable or that one’s own research is valuable or some such, rather than facing the harsh truth that much research (possibly one’s own) is predictably-in-advance worthless.
It is probably often motivated in that way, though interestingly something I had in mind while writing my comment was something like an opposite bias (likewise not accusing you specifically of it). In that in rationalist/EA circles it sometimes feels like everyone (myself included) wants to do the meta-research, the synthesizing across disciplines, the solving of the key bottlenecks etc. and there’s a relative lack of interest in the object-level research, the stamp collecting, the putting things in place that’s a prerequisite for understanding and solving the key bottlenecks. In a way that puts the meta stuff as the highest good while glossing over the fact that the meta stuff only works if someone else has done the basic research it builds on first.
Now your post wasn’t framed in terms of meta-research vs. object-level research nor of theory-building vs. stamp-collecting or anything like that, so this criticism doesn’t apply to your post as a whole. But I think the algorithm of “try to ensure that your research is valuable and not useless” that I was responding to, while by itself sensible, can easily be (mis?)applied in a way that causes one to gravitate toward more meta/theory stuff. (Especially if people do the often-tempting move of using the prestige of a discovery as a proxy for its usefulness.) This can then, I think, increase the probability that the individual gets to claim credit for a shiny-looking discovery while reducing the probability that they’ll do something more generally beneficial.
Toy model: suppose that each empirical result has some probability of being useful. For every U useful empirical results, there are T theoretical discoveries to be made that generalize across those empirical results. Suppose that useful empirical results give you a little prestige while theoretical discoveries give you a lot of prestige, and each scientist can work on either empiricism or theory. Given enough empirical findings, each theorist has some probability of making a theoretical discovery over time.
Then past a certain point, becoming a theorist will not make it significantly more likely that science overall advances (as the number of theoretical discoveries to be made is bounded by the number of empirical findings and some other theorist would have been likely to make the same discovery), but it does increase that theorist’s personal odds of getting a lot of prestige. At the same time, society might be better off if more people were working on empirical findings, as that allowed more theoretical discoveries to be made.
Of course this is a pretty general and abstract argument and it only applies if the balance of theorists vs. empiricists is in fact excessively tilted toward the theorists. I don’t know whether that’s true, I could easily imagine that the opposite was. (And again it’s not directly related to most of what you were saying in your post, though there’s a possible analogous argument to be made about whether there was any predictably useful work left to be done in the first place once Alice, The Very General Helper, and The One Who Actually Thought This Through A Bit were already working on their respective approaches.)
There’s a world of difference between “let’s just continue doing this research project on something obscure with no theory of impact because penicillin” and “this is more likely than not to become irrelevant in 18 months time, but if it works, it will be a game-changer”.
Robustness and generalizability are subcomponents of the expected value of your work/research. If you think that these are neglected, and that your field is too focused on the “impact” components of EV, i.e. there are too many moon shots, please clarify that, but your analogy fails to make this argument.
As it is, I suspect that optimizing for robust generalizability is a sure-fire way of ensuring that most people become “very general helpers”, which seems like a very harmful thing to promote.
Though it also feels worth noting that while individually unsatisfying, it’s not necessarily collectively wrong for a lot of people to work on subproblems that will turn out to be completely worthless. If one of them turns out to unexpectedly solve the main bottleneck after all, or if several of them do end up making small progress that lets someone else solve the main bottleneck, all of the “wasted” work may in fact have bought us the useful bits that you couldn’t have identified in advance.
Often when I’ve had a hypothesis about something that interests me, I’ve been happy that there has been *so much* scientific research done on various topics, many of them seemingly insignificant. While most of it is of little interest to me, the fact that there’s so much of it means that there’s often some prior work on topics that do interest me. And vice versa—much of the work that I find uninteresting, will probably be a useful puzzle piece in someone else’s research. (Even if a lot of it is also genuinely useless for everyone, but that’s the price we pay for getting some of the more useful bits.)
That sort of reasoning makes sense insofar as it’s hard to predict which small pieces will be useful. And while that is hard to some extent, it is not full we-just-have-no-idea-so-use-a-maxent-prior hard. There is plenty of work (including lots of research which people sink their lives into today) which will predictably-in-advance be worthless. And robust generalizability is the main test I know of for that purpose.
Applying this to your own argument:
It will predictably and systematically be the robustly generalizable things which are relevant to other people in unexpected ways.
Now to get a little more harsh...
Without necessarily accusing Kaj specifically, this general type of argument feels motivated to me. It feels like willful ignorance, like sticking one’s head in the sand and ignoring the available information, because one wants to believe that All Research is Valuable or that one’s own research is valuable or some such, rather than facing the harsh truth that much research (possibly one’s own) is predictably-in-advance worthless.
It is probably often motivated in that way, though interestingly something I had in mind while writing my comment was something like an opposite bias (likewise not accusing you specifically of it). In that in rationalist/EA circles it sometimes feels like everyone (myself included) wants to do the meta-research, the synthesizing across disciplines, the solving of the key bottlenecks etc. and there’s a relative lack of interest in the object-level research, the stamp collecting, the putting things in place that’s a prerequisite for understanding and solving the key bottlenecks. In a way that puts the meta stuff as the highest good while glossing over the fact that the meta stuff only works if someone else has done the basic research it builds on first.
Now your post wasn’t framed in terms of meta-research vs. object-level research nor of theory-building vs. stamp-collecting or anything like that, so this criticism doesn’t apply to your post as a whole. But I think the algorithm of “try to ensure that your research is valuable and not useless” that I was responding to, while by itself sensible, can easily be (mis?)applied in a way that causes one to gravitate toward more meta/theory stuff. (Especially if people do the often-tempting move of using the prestige of a discovery as a proxy for its usefulness.) This can then, I think, increase the probability that the individual gets to claim credit for a shiny-looking discovery while reducing the probability that they’ll do something more generally beneficial.
Toy model: suppose that each empirical result has some probability of being useful. For every U useful empirical results, there are T theoretical discoveries to be made that generalize across those empirical results. Suppose that useful empirical results give you a little prestige while theoretical discoveries give you a lot of prestige, and each scientist can work on either empiricism or theory. Given enough empirical findings, each theorist has some probability of making a theoretical discovery over time.
Then past a certain point, becoming a theorist will not make it significantly more likely that science overall advances (as the number of theoretical discoveries to be made is bounded by the number of empirical findings and some other theorist would have been likely to make the same discovery), but it does increase that theorist’s personal odds of getting a lot of prestige. At the same time, society might be better off if more people were working on empirical findings, as that allowed more theoretical discoveries to be made.
Of course this is a pretty general and abstract argument and it only applies if the balance of theorists vs. empiricists is in fact excessively tilted toward the theorists. I don’t know whether that’s true, I could easily imagine that the opposite was. (And again it’s not directly related to most of what you were saying in your post, though there’s a possible analogous argument to be made about whether there was any predictably useful work left to be done in the first place once Alice, The Very General Helper, and The One Who Actually Thought This Through A Bit were already working on their respective approaches.)
There’s a world of difference between “let’s just continue doing this research project on something obscure with no theory of impact because penicillin” and “this is more likely than not to become irrelevant in 18 months time, but if it works, it will be a game-changer”.
Robustness and generalizability are subcomponents of the expected value of your work/research. If you think that these are neglected, and that your field is too focused on the “impact” components of EV, i.e. there are too many moon shots, please clarify that, but your analogy fails to make this argument.
As it is, I suspect that optimizing for robust generalizability is a sure-fire way of ensuring that most people become “very general helpers”, which seems like a very harmful thing to promote.
<irony>Robustly generalizible like noticing that bacteria aren’t growing next to a certain kind of mold that contaminated your petri dish or that photographic film is getting fogged when there’s no obvious source of light?</irony>