How are you looking at “researchers” vs. “engineers”? At some organizations, i.e. Redwood, the boundary is very fuzzy—there isn’t a sharp delineation between anyone whose job it is to primarily “think of ideas”, vs. “implement ideas that researchers come up with”, so it seems reasonable to count most of their technical staff as researchers.
FAR, on the other hand, does have separate job titles for “Research Scientist” (4 people) vs. “Research Engineer” (5 people), though they do also say they “expect everyone on the project to help shape the research direction”.
Some of the other numbers seem like overestimates.
CHAI has 2 researchers and 6 research fellows, and only 2 (maybe 3) of the research fellows are doing anything recognizable as alignment research. (Not extremely confident; didn’t spend a lot of time digging for details for those that didn’t have websites. But generally not optimistic.) One of the researcher is Andrew Critch, who is one of the two people at Encultured. If you throw in Stuart Russell that’s maybe 6 people, not 30.
FHI has 2 people in their AI Safety Research Group. There are also a couple people in their macrostrategy research group it wouldn’t be crazy to count. Everybody else listed on the page either isn’t working on technical Alignment research or is doing so under another org also listed here. So maybe 4 people, rather than 10?
I don’t have very up-to-date information but I would pretty surprised if MIRI had 15 full-time research staff right now.
Also, I think that every single person I counted above has at least a LessWrong account, and most also have Alignment Forum accounts, so a good chunk are probably double-counted.
On the other hand, there are a number of people going through SERI MATS who probably weren’t counted; most of them will have LessWrong accounts but probably not Alignment Forum accounts (yet).
I’d be very happy to learn that there were 5 people at Meta doing something recognizable as alignment research; the same for Google Brain. Do you have any more info on those?
I added a new section in the introduction named ‘Definitions’ to define some commonly-used terms in the post and decrease ambiguity. To answer your question, research engineers and research scientists would be in the technical AI safety research category.
I re-estimating the number of researchers at the organizations you mentioned and came up with the following numbers:
From reading their website, CHAI has a lot graduate students and interns and I would consider them to be full-time researchers. My previous estimate for CHAI is 10-30-60 (low-estimate-high) and I changed it to 5-25-50 in light of this new information. My estimate is less than 40 to be conservative and also because I doubt all of these researchers are working full-time at CHAI. Also, some of them have probably been counted in the Alignment Forum total.
FHI
Information from their website: - AI safety research group: 2 - AI governance researchers: 1 - Research Scholars Programme + DPhil Scholars and Affiliates: 9
- Total: 12
Change in estimate: 10-10-40 → 5-10-30
MIRI - 1 leader (Nate Soares) - 9 research staff
Change in estimate: 10-15-30 → 5-10-20
“Also, I think that every single person I counted above has at least a LessWrong account, and most also have Alignment Forum accounts, so a good chunk are probably double-counted.”
I analyzed the people who posted AI safety posts on LessWrong and found that only 15% also had Alignment Forum accounts. I avoided double-counting by subtracting the LessWrong users who also have an Alignment Forum account from the LessWrong total.
For SERI MATS, I’m guessing that some of those people will be counted in the AF count. I also added an ‘Other’ row for other groups I didn’t include in the table.
I decided to delete the Google Brain and Meta entries because I have very little information about them.
Interesting analysis. Some questions and notes:
How are you looking at “researchers” vs. “engineers”? At some organizations, i.e. Redwood, the boundary is very fuzzy—there isn’t a sharp delineation between anyone whose job it is to primarily “think of ideas”, vs. “implement ideas that researchers come up with”, so it seems reasonable to count most of their technical staff as researchers.
FAR, on the other hand, does have separate job titles for “Research Scientist” (4 people) vs. “Research Engineer” (5 people), though they do also say they “expect everyone on the project to help shape the research direction”.
Some of the other numbers seem like overestimates.
CHAI has 2 researchers and 6 research fellows, and only 2 (maybe 3) of the research fellows are doing anything recognizable as alignment research. (Not extremely confident; didn’t spend a lot of time digging for details for those that didn’t have websites. But generally not optimistic.) One of the researcher is Andrew Critch, who is one of the two people at Encultured. If you throw in Stuart Russell that’s maybe 6 people, not 30.
FHI has 2 people in their AI Safety Research Group. There are also a couple people in their macrostrategy research group it wouldn’t be crazy to count. Everybody else listed on the page either isn’t working on technical Alignment research or is doing so under another org also listed here. So maybe 4 people, rather than 10?
I don’t have very up-to-date information but I would pretty surprised if MIRI had 15 full-time research staff right now.
Also, I think that every single person I counted above has at least a LessWrong account, and most also have Alignment Forum accounts, so a good chunk are probably double-counted.
On the other hand, there are a number of people going through SERI MATS who probably weren’t counted; most of them will have LessWrong accounts but probably not Alignment Forum accounts (yet).
I’d be very happy to learn that there were 5 people at Meta doing something recognizable as alignment research; the same for Google Brain. Do you have any more info on those?
Thanks for the feedback.
I added a new section in the introduction named ‘Definitions’ to define some commonly-used terms in the post and decrease ambiguity. To answer your question, research engineers and research scientists would be in the technical AI safety research category.
I re-estimating the number of researchers at the organizations you mentioned and came up with the following numbers:
CHAI
- 2 researchers
- 6 researchers
- 21 graduate students
- 11 interns
Total: 40
From reading their website, CHAI has a lot graduate students and interns and I would consider them to be full-time researchers. My previous estimate for CHAI is 10-30-60 (low-estimate-high) and I changed it to 5-25-50 in light of this new information. My estimate is less than 40 to be conservative and also because I doubt all of these researchers are working full-time at CHAI. Also, some of them have probably been counted in the Alignment Forum total.
FHI
Information from their website:
- AI safety research group: 2
- AI governance researchers: 1
- Research Scholars Programme + DPhil Scholars and Affiliates: 9
- Total: 12
Change in estimate: 10-10-40 → 5-10-30
MIRI
- 1 leader (Nate Soares)
- 9 research staff
Change in estimate: 10-15-30 → 5-10-20
I analyzed the people who posted AI safety posts on LessWrong and found that only 15% also had Alignment Forum accounts. I avoided double-counting by subtracting the LessWrong users who also have an Alignment Forum account from the LessWrong total.
For SERI MATS, I’m guessing that some of those people will be counted in the AF count. I also added an ‘Other’ row for other groups I didn’t include in the table.
I decided to delete the Google Brain and Meta entries because I have very little information about them.