I think forward-deployed research engineers (FDREs) should mainly work with independent nonprofits / academic labs who have a good theory of change but wouldn’t have capital to normally attract RE talent by themselves
I think there should be conditions attached to the use of such research engineers, e.g. working towards an agreed agenda, and making sure to publish all findings in an unbiased way.
In practice FDREs should be viewed as another type of “grant”, which is somewhat-fungible (but not totally fungible) with monetary grants, and doled out by equivalent grantmaking organizations
I believe they have the highest value in model labs
I disagree with this, for the following reasons
Even assuming that catastrophic risks are the main category of risk, the best way to address this might not be by accelerating labs. I subscribe to Geoffrey Irving’s view that more diverse bets are needed to solve alignment.
It’s not clear that having these engineers be at labs would differentially accelerate safety, and indeed it might accelerate capabilities instead (e.g. research is dual use, or labs can respond by allocating more of their own engineer-hours to capabilities instead of safety / alignment research)
I think labs are already able to attract sufficient talent so they don’t need additional help from the nonprofit side
my understanding is that theres a higher focus on alignment researchers rather than security researchers (at least based on my read on programs like MATS)
I also disagree with this, MATS seems to be diversifying quite hard with their recent batches, and I would now say that (prosaic alignment, conceptual alignment, cybersecurity, policy / governance, biosecurity, and founding / field-building) are all represented to some extent
I think forward-deployed research engineers (FDREs) should mainly work with independent nonprofits / academic labs who have a good theory of change but wouldn’t have capital to normally attract RE talent by themselves
I think there should be conditions attached to the use of such research engineers, e.g. working towards an agreed agenda, and making sure to publish all findings in an unbiased way.
In practice FDREs should be viewed as another type of “grant”, which is somewhat-fungible (but not totally fungible) with monetary grants, and doled out by equivalent grantmaking organizations
I disagree with this, for the following reasons
Even assuming that catastrophic risks are the main category of risk, the best way to address this might not be by accelerating labs. I subscribe to Geoffrey Irving’s view that more diverse bets are needed to solve alignment.
It’s not clear that having these engineers be at labs would differentially accelerate safety, and indeed it might accelerate capabilities instead (e.g. research is dual use, or labs can respond by allocating more of their own engineer-hours to capabilities instead of safety / alignment research)
I think labs are already able to attract sufficient talent so they don’t need additional help from the nonprofit side
I also disagree with this, MATS seems to be diversifying quite hard with their recent batches, and I would now say that (prosaic alignment, conceptual alignment, cybersecurity, policy / governance, biosecurity, and founding / field-building) are all represented to some extent