Hi Anna, this looks like a great initiative! I think 200 papers seems quite small though. As an example, did you include all papers with contributions from MATS (~185 papers), EleutherAI, Apart, PIBBSS, Pivotal, SPAR, ERA, LASR, MARS, etc.? My impression is that a large amount of AI safety research is published in collaboration with nonprofit fellowship programs that borrow mentors from other institutions, which might help explain the apparent centrality of universities.
Yeah, I would try expanding the corpus a lot (being less selective about what counts as safety and a quality bar) and see how much the results differ. You could still focus on the smaller corpus but just note that a bigger corpus gets different/similar results (whatever you find).
I agree that comparing our results with a larger, less selective corpus would be an important robustness check, and we do plan on doing this eventually. We considered defining quantitative criteria for a corpus in the first place but were worried that keywords would give us a distorted result in a somewhat fragmented, interdisciplinary space with a broad range of participants.
Thanks for these! We will review every link and check our coverage. Are you pointing any of these organizations out specifically because after looking at the dataset, they strike you as underrepresented?
It’s important to consider that university centrality might be an artifact because non-profit fellowships borrow mentors from academia and industry, or fellows might list their graduate programs as their affiliation. We need to think carefully about whether fellowship programs like MATS create a coding ambiguity where work done in an industry-adjacent context gets attributed to academic institutions. We’ll check the prevalence of this and look for a way to represent how many of our multiply affiliated authors are mentors or have fellowship connections.
Thanks for replying! I listed these organizations because they all maintain up-to-date repositories of papers they contributed to. If you added all papers linked there (and I think you should, as they are all AI safety papers), I suspect you would have ~400-500 papers, many of which would not be included in your initial 200!
If I were running this project, I would additionally scrape papers from the websites of all the orgs listed on the AI safety map (they have a spreadsheet of orgs) and 80,000 Hours org list.
Hi Anna, this looks like a great initiative! I think 200 papers seems quite small though. As an example, did you include all papers with contributions from MATS (~185 papers), EleutherAI, Apart, PIBBSS, Pivotal, SPAR, ERA, LASR, MARS, etc.? My impression is that a large amount of AI safety research is published in collaboration with nonprofit fellowship programs that borrow mentors from other institutions, which might help explain the apparent centrality of universities.
Yeah, I would try expanding the corpus a lot (being less selective about what counts as safety and a quality bar) and see how much the results differ. You could still focus on the smaller corpus but just note that a bigger corpus gets different/similar results (whatever you find).
I agree that comparing our results with a larger, less selective corpus would be an important robustness check, and we do plan on doing this eventually. We considered defining quantitative criteria for a corpus in the first place but were worried that keywords would give us a distorted result in a somewhat fragmented, interdisciplinary space with a broad range of participants.
Thanks for these! We will review every link and check our coverage. Are you pointing any of these organizations out specifically because after looking at the dataset, they strike you as underrepresented?
It’s important to consider that university centrality might be an artifact because non-profit fellowships borrow mentors from academia and industry, or fellows might list their graduate programs as their affiliation. We need to think carefully about whether fellowship programs like MATS create a coding ambiguity where work done in an industry-adjacent context gets attributed to academic institutions. We’ll check the prevalence of this and look for a way to represent how many of our multiply affiliated authors are mentors or have fellowship connections.
Thanks for replying! I listed these organizations because they all maintain up-to-date repositories of papers they contributed to. If you added all papers linked there (and I think you should, as they are all AI safety papers), I suspect you would have ~400-500 papers, many of which would not be included in your initial 200!
If I were running this project, I would additionally scrape papers from the websites of all the orgs listed on the AI safety map (they have a spreadsheet of orgs) and 80,000 Hours org list.
Also note this database.