what could be done to change those job/funding incentives? are existing grantmakers too timid? is there room to start a new grantmaking org that makes more ambitious bets?
I’m not very clued in to the current grantmaker space. But I know that top researchers in interp are queried far less about funding proposals than what they can handle. I expect it’s the same for other fields.
My thoughts on this are pretty uncertain but here’s some semi hot takes. Listed as they come to me in no particular attempt to make a coherent case for anything.
* We want ambitious but also good research. Easier said than done obviously. * Having an ambitious research program is anti-correlated with experience. This is too bad, because research experience is extremely important and likely undervalued or hard to value (as is the case with the ever elusive taste, which I think one can develop through experience). * Ambitious research is not worth much without all the other things that make good research good. * It’s not obvious to me the issue is primarily one of incentives. * The main thing one should do to get more people doing ambitious research is to do good ambitious research yourself, and to show by example what good ambitious research looks like, and convince others through object level things that this is a good path to take. * It’s harder to do good ambitious research than it is to do good research that’s less ambitious. That’s a real tradeoff! * I think funding opportunities are there for ambitious research to happen, but the bar is higher for it to be funded, and that seems reasonable? * I’m still unclear on what people actually mean when they say ambitious research. For instance, sometimes “ambitious interp” is used for things I don’t personally find all that ambitious. I have no doubt that things I think are ambitious, others find quite pedestrian. Maybe it’s a taste thing.
My understanding is that ‘ambitious’ proposals are often highly illegible, such that only a few dozen humans are equipped to seriously evaluate them (even with substantial effort), and those people often have strong opinions that weigh unfavorably on their appraisal of proposals, as well as having directions they’re more excited about that weigh heavily on their time.
Like, when I imagine the people I’d like to see evaluating the proposals, it’s sort of the same 10-20 people I wish were doing everything, because research taste is extremely scarce, and perhaps the most important resource in grant making.
It’s also not very institutionally scalable.
(80 percent confidence; I’ve been around a lot of grant making and fieldbuilding, but haven’t been In The Room for an explicit funding decision regarding projects outside of the dominant, lab-supported ML paradigm of safety.)
Fieldbuilders are trying this already (eg courting experienced scientists in other fields, building bridges to academia, etc). It’s not really clear to me how well this works (or how often research taste generalizes cross-field, since it’s a heavily context-dependent skill).
Further, the most likely kind of person to poach successfully from another field is an MLE, and they’re more likely to pursue various ML approaches that I expect our OP would consider ‘unambitious’.
MATS, at least through 2024 (and maybe still; I don’t know), put a lot of emphasis on trying to help people develop research taste. I think results were sort of mixed, because this is an extremely difficult thing to teach, or even describe in a way that someone who doesn’t feel motivated to learn it already will understand.
Fieldbuilders are trying this already (eg courting experienced scientists in other fields, building bridges to academia, etc). It’s not really clear to me how well this works (or how often research taste generalizes cross-field, since it’s a heavily context-dependent skill).
I don’t think research taste in other fields implies very much alignment taste (e.g.).
MATS, at least through 2024 (and maybe still; I don’t know), put a lot of emphasis on trying to help people develop research taste. I think results were sort of mixed, because this is an extremely difficult thing to teach, or even describe in a way that someone who doesn’t feel motivated to learn it already will understand.
I think it failed because there weren’t good enough feedback loops to assess whether people got any better at taste.
I agree with both of these things (although I’m not sure I would say MATS failed, rather than ‘it’s going worse than one might hope, and feedback loops are one major reason for that’).
So what did you mean by ‘getting people with that sort of research taste’? Like, somehow incentivizing people in that small group of 10-20 or so to spend more time evaluating grants?
what could be done to change those job/funding incentives? are existing grantmakers too timid? is there room to start a new grantmaking org that makes more ambitious bets?
I’m not very clued in to the current grantmaker space. But I know that top researchers in interp are queried far less about funding proposals than what they can handle. I expect it’s the same for other fields.
My thoughts on this are pretty uncertain but here’s some semi hot takes. Listed as they come to me in no particular attempt to make a coherent case for anything.
* We want ambitious but also good research. Easier said than done obviously.
* Having an ambitious research program is anti-correlated with experience. This is too bad, because research experience is extremely important and likely undervalued or hard to value (as is the case with the ever elusive taste, which I think one can develop through experience).
* Ambitious research is not worth much without all the other things that make good research good.
* It’s not obvious to me the issue is primarily one of incentives.
* The main thing one should do to get more people doing ambitious research is to do good ambitious research yourself, and to show by example what good ambitious research looks like, and convince others through object level things that this is a good path to take.
* It’s harder to do good ambitious research than it is to do good research that’s less ambitious. That’s a real tradeoff!
* I think funding opportunities are there for ambitious research to happen, but the bar is higher for it to be funded, and that seems reasonable?
* I’m still unclear on what people actually mean when they say ambitious research. For instance, sometimes “ambitious interp” is used for things I don’t personally find all that ambitious. I have no doubt that things I think are ambitious, others find quite pedestrian. Maybe it’s a taste thing.
My understanding is that ‘ambitious’ proposals are often highly illegible, such that only a few dozen humans are equipped to seriously evaluate them (even with substantial effort), and those people often have strong opinions that weigh unfavorably on their appraisal of proposals, as well as having directions they’re more excited about that weigh heavily on their time.
Like, when I imagine the people I’d like to see evaluating the proposals, it’s sort of the same 10-20 people I wish were doing everything, because research taste is extremely scarce, and perhaps the most important resource in grant making.
It’s also not very institutionally scalable.
(80 percent confidence; I’ve been around a lot of grant making and fieldbuilding, but haven’t been In The Room for an explicit funding decision regarding projects outside of the dominant, lab-supported ML paradigm of safety.)
Given this, it seems like we probably should invest more resources in getting people with that sort of research taste.
Fieldbuilders are trying this already (eg courting experienced scientists in other fields, building bridges to academia, etc). It’s not really clear to me how well this works (or how often research taste generalizes cross-field, since it’s a heavily context-dependent skill).
Further, the most likely kind of person to poach successfully from another field is an MLE, and they’re more likely to pursue various ML approaches that I expect our OP would consider ‘unambitious’.
MATS, at least through 2024 (and maybe still; I don’t know), put a lot of emphasis on trying to help people develop research taste. I think results were sort of mixed, because this is an extremely difficult thing to teach, or even describe in a way that someone who doesn’t feel motivated to learn it already will understand.
I don’t think research taste in other fields implies very much alignment taste (e.g.).
I think it failed because there weren’t good enough feedback loops to assess whether people got any better at taste.
I agree with both of these things (although I’m not sure I would say MATS failed, rather than ‘it’s going worse than one might hope, and feedback loops are one major reason for that’).
So what did you mean by ‘getting people with that sort of research taste’? Like, somehow incentivizing people in that small group of 10-20 or so to spend more time evaluating grants?
I think the hard part of getting research taste is good feedback loops, but on the object level, e.g. this seems like a promising direction.
I think funding incentives are really quite fucked up. I roughly stand behind my summary of funding incentives in this comment (not perfectly, but roughly): https://www.lesswrong.com/posts/wn5jTrtKkhspshA4c/michaeldickens-s-shortform?commentId=zoBMvdMAwpjTEY4st