As a data point, I did ARENA 2.0 and there’s a good chance I work in capabilities next year. I would mostly prefer to work in alignment and am applying to several different labs, but there is currently very little funding for alignment work, such that I’ve been unfunded or partially unfunded for several months. If this continues to be the case, I’ll need to get a job somewhere, and I have various reasons why working in capabilities is better than quant trading or other non-ML jobs.
I think that giving people enough ML skills to work on capabilities is basically unavoidable if you’re producing competent technical researchers, and Callum took reasonable steps, listed in his sibling comment, to make people more likely to work in alignment.
As a data point, I did ARENA 2.0 and there’s a good chance I work in capabilities next year. I would mostly prefer to work in alignment and am applying to several different labs, but there is currently very little funding for alignment work, such that I’ve been unfunded or partially unfunded for several months. If this continues to be the case, I’ll need to get a job somewhere, and I have various reasons why working in capabilities is better than quant trading or other non-ML jobs.
I think that giving people enough ML skills to work on capabilities is basically unavoidable if you’re producing competent technical researchers, and Callum took reasonable steps, listed in his sibling comment, to make people more likely to work in alignment.