I was planning to make a post at some point with some advice that’s closely related to this post but I will share it here as a preview. Take note that I don’t yet have strong evidence that my work is good or has mattered (and I was going to write a full post once I had more evidence for that). I think Richard’s advice above is really good and I’ll try to take some of the ideas more on board with my own work.
Last year I quit my job and upskilled for 6 months and now I’m doing independent research which might turn out to be valuable. Regardless of its value, I’ve learnt a lot and it’s created many opportunities for me. I went to EAG and Richard’s talk there and a conversation later in a group where he was talking about this mentorship constraint deal. This left a strong impression on me leading me to take some degree of pride in my attempts to be independent and not rely as strongly on individual mentorship. However, there are just a bunch of caveats/perspectives that I have currently which relate to this.
All of these relate to empirical alignment research and not governance or other forms of research. I’m mostly focussed on providing advice for how to be more productive independently of other people but that shouldn’t be your preference and I suspect people are more productive at orgs/in groups.
So a bunch of ideas on the topic:
Why the focus on independent research?
I think it’s really weird how we have this thing in the alignment community and I just want to comment on that first. The idea that people can just go off on their own and be productive I think is kinda uncommon.
This community values agency. In practice, agency is the ability to make decisions for yourself about what you want and how to achieve it. Getting good at having agency both makes good researchers and good research engineers. HPMOR helped me understand agency better.
I have no first hand knowledge of the inside of orgs like DeepMind or Anthropic but I suspect people with agency are generally considered better hires. It’s not like orgs would say “we could hire this person but we want them to do what they’re told so let’s hire someone with little evidence of working independently”. Rather, my guess is they select for people who are capable of being self-directed work and who grow spontaneously as a result on attempting hard things and learning.
Getting ready to contribute:
There are a variety of ways to upskill without doing stuff like a PhD (as research says above). Programs like ARENA, SERI-MATS, SPAR etc. My sense is that once people realise that working without feedback is hard, they will gravitate strongly toward more empirical research areas, especially those that can be done at small scale (aka, MI) and which there are existing tools (aka MI) and examples of investigations with reasoned paths to impact (aka MI). However, there are likely other empirical areas which provide feedback and are doable that may/may not have these properties and searching for more seems like a good idea to me.
Get good. (especially if you’re fresh out of college) Struggling with implementing things / understanding stuff and identifying problems can all be alleviated by working on your skills. There’s lots of open source code which show good patterns but also lots of people who have good technical skills but aren’t necessarily the research mentors we are constrained by who you can engage with. Talk to people who are good engineers and find out how they operate. It’ll be stuff like having a good IDE and testing your code.
Start slow. Contributing to open source projects such as TransformerLens is good. I’ve been helping out with it and it seems like a good way for lots of people to dip their toe in.
Doing research without a mentor is very hard for many obvious reasons. Things you can do to make it easier:
While talking to people such at EAGs can be helpful, my sense is most good advice just exists on the forum. I recommend rereading such advice periodically and predict you will grok why people make suggestions more if you are stuck in your own research and have challenges then before.
Focus on fast feedback cycles. Try to avoid situations where you don’t know if something is working for a long time. This is different to whether you know if it’s valuable or not.
Be prepared to drop things or change your path, but don’t abandon work because it’s hard. It feels like a special kind of wisdom/insight to make these calls and I think you need to work hard at trying to get better at this over time.
Have good tooling but don’t let building the tooling take over.
Allow yourself to focus. There is a time to work out why you are doing what you are doing and there are other times you just need to do the work.
Study! Engaging with fundamental topics like linear algebra or deep learning theory is hugely important. Without colleagues or mentors it is a meaningful constraint on your output when you don’t know any given thing that might be relevant. This is tricky because there’s a lot to study. I think engage with the basics and be consistent. Mathematics for ML textbook is great. GoodFellow Deep Learning textbook is also recommended.
Read related literature. Like with more basic knowledge, lack of knowledge of relevant literature can cause you to waste time/effort. I have a spreadsheet which describes all the models that are kinda similar to mine and how they were trained and what was done with them.
Find less correlated ideas/takes: Stephen Casper’s Engineering interpreatibility sequence is a good example of the kind of thing people doing independent work should read. It shakes you out of the “everything we do here makes sense and is obvious perspective” which is extra easy to fall into when you work on your own. There might be equivalent posts in other areas.
Possibly the quirkiest thing I do these days is roleplay characters in my head “the engineer”, “the scientist”, “the manager” and the “outsider” who help me balance different priorities when making decisions about my work. I find this fun and useful and since I literally write meeting notes, GPT4 can stand in for each of them which is pretty cool and useful for generating ideas. The “other”, a less obvious team member, represents someone who doesn’t privilege the project or existing decisions. This helps me try to channel a helpful adversarial perspective (see previous point).
Thanks Richard for this post and prior advice!
I was planning to make a post at some point with some advice that’s closely related to this post but I will share it here as a preview. Take note that I don’t yet have strong evidence that my work is good or has mattered (and I was going to write a full post once I had more evidence for that). I think Richard’s advice above is really good and I’ll try to take some of the ideas more on board with my own work.
Last year I quit my job and upskilled for 6 months and now I’m doing independent research which might turn out to be valuable. Regardless of its value, I’ve learnt a lot and it’s created many opportunities for me. I went to EAG and Richard’s talk there and a conversation later in a group where he was talking about this mentorship constraint deal. This left a strong impression on me leading me to take some degree of pride in my attempts to be independent and not rely as strongly on individual mentorship. However, there are just a bunch of caveats/perspectives that I have currently which relate to this.
All of these relate to empirical alignment research and not governance or other forms of research. I’m mostly focussed on providing advice for how to be more productive independently of other people but that shouldn’t be your preference and I suspect people are more productive at orgs/in groups.
So a bunch of ideas on the topic:
Why the focus on independent research?
I think it’s really weird how we have this thing in the alignment community and I just want to comment on that first. The idea that people can just go off on their own and be productive I think is kinda uncommon.
This community values agency. In practice, agency is the ability to make decisions for yourself about what you want and how to achieve it. Getting good at having agency both makes good researchers and good research engineers. HPMOR helped me understand agency better.
I have no first hand knowledge of the inside of orgs like DeepMind or Anthropic but I suspect people with agency are generally considered better hires. It’s not like orgs would say “we could hire this person but we want them to do what they’re told so let’s hire someone with little evidence of working independently”. Rather, my guess is they select for people who are capable of being self-directed work and who grow spontaneously as a result on attempting hard things and learning.
Getting ready to contribute:
There are a variety of ways to upskill without doing stuff like a PhD (as research says above). Programs like ARENA, SERI-MATS, SPAR etc. My sense is that once people realise that working without feedback is hard, they will gravitate strongly toward more empirical research areas, especially those that can be done at small scale (aka, MI) and which there are existing tools (aka MI) and examples of investigations with reasoned paths to impact (aka MI). However, there are likely other empirical areas which provide feedback and are doable that may/may not have these properties and searching for more seems like a good idea to me.
Get good. (especially if you’re fresh out of college) Struggling with implementing things / understanding stuff and identifying problems can all be alleviated by working on your skills. There’s lots of open source code which show good patterns but also lots of people who have good technical skills but aren’t necessarily the research mentors we are constrained by who you can engage with. Talk to people who are good engineers and find out how they operate. It’ll be stuff like having a good IDE and testing your code.
Start slow. Contributing to open source projects such as TransformerLens is good. I’ve been helping out with it and it seems like a good way for lots of people to dip their toe in.
Doing research without a mentor is very hard for many obvious reasons. Things you can do to make it easier:
While talking to people such at EAGs can be helpful, my sense is most good advice just exists on the forum. I recommend rereading such advice periodically and predict you will grok why people make suggestions more if you are stuck in your own research and have challenges then before.
Focus on fast feedback cycles. Try to avoid situations where you don’t know if something is working for a long time. This is different to whether you know if it’s valuable or not.
Be prepared to drop things or change your path, but don’t abandon work because it’s hard. It feels like a special kind of wisdom/insight to make these calls and I think you need to work hard at trying to get better at this over time.
Have good tooling but don’t let building the tooling take over.
Allow yourself to focus. There is a time to work out why you are doing what you are doing and there are other times you just need to do the work.
Study! Engaging with fundamental topics like linear algebra or deep learning theory is hugely important. Without colleagues or mentors it is a meaningful constraint on your output when you don’t know any given thing that might be relevant. This is tricky because there’s a lot to study. I think engage with the basics and be consistent. Mathematics for ML textbook is great. GoodFellow Deep Learning textbook is also recommended.
Read related literature. Like with more basic knowledge, lack of knowledge of relevant literature can cause you to waste time/effort. I have a spreadsheet which describes all the models that are kinda similar to mine and how they were trained and what was done with them.
Find less correlated ideas/takes: Stephen Casper’s Engineering interpreatibility sequence is a good example of the kind of thing people doing independent work should read. It shakes you out of the “everything we do here makes sense and is obvious perspective” which is extra easy to fall into when you work on your own. There might be equivalent posts in other areas.
Possibly the quirkiest thing I do these days is roleplay characters in my head “the engineer”, “the scientist”, “the manager” and the “outsider” who help me balance different priorities when making decisions about my work. I find this fun and useful and since I literally write meeting notes, GPT4 can stand in for each of them which is pretty cool and useful for generating ideas. The “other”, a less obvious team member, represents someone who doesn’t privilege the project or existing decisions. This helps me try to channel a helpful adversarial perspective (see previous point).
I hope this is useful for people!
Thanks, I think this comment and the subsequent post will be very useful for me!