Citation-network analysis would be a natural complement here. 200 papers is a narrow base, and the corpus is built on an implicit selection criterion of “what people on LW/CHAI/Hendrycks/Barak treat as canon,” that’s worth stating explicitly, since it shapes the conclusions. “Concrete Problems in AI Safety,” one of the included papers alone has ~5,000 citing papers on Google Scholar; even after aggressive filtering for actual safety relevance, that’s a very different (and much larger) population than the curated canon. I’m curious to see what the wider network would look like.
Every purposive sample is opinionated, which is why we’re asking for community feedback. Do you have other figures or institutions in mind we should include that would balance our selection?
We seriously considered incorporating citation analysis, but couldn’t figure out a way to execute it for this project in a way that makes sense. Overlaying the directionality of citations with co-authorships would be fascinating, and a citation-based corpus would capture a different and much larger population, but it’s tricky. In a fast-moving field of posts and preprints it would be challenging to find a convincing formula for time-weighting the number of citations papers receive, and the valence of a citation is much more ambiguous than publicly acknowledged co-authorship. We ultimately chose co-authorship because it’s easier to interpret, and because we are focused on collaboration structure rather than influence.
Citation-network analysis would be a natural complement here. 200 papers is a narrow base, and the corpus is built on an implicit selection criterion of “what people on LW/CHAI/Hendrycks/Barak treat as canon,” that’s worth stating explicitly, since it shapes the conclusions. “Concrete Problems in AI Safety,” one of the included papers alone has ~5,000 citing papers on Google Scholar; even after aggressive filtering for actual safety relevance, that’s a very different (and much larger) population than the curated canon. I’m curious to see what the wider network would look like.
Every purposive sample is opinionated, which is why we’re asking for community feedback. Do you have other figures or institutions in mind we should include that would balance our selection?
We seriously considered incorporating citation analysis, but couldn’t figure out a way to execute it for this project in a way that makes sense. Overlaying the directionality of citations with co-authorships would be fascinating, and a citation-based corpus would capture a different and much larger population, but it’s tricky. In a fast-moving field of posts and preprints it would be challenging to find a convincing formula for time-weighting the number of citations papers receive, and the valence of a citation is much more ambiguous than publicly acknowledged co-authorship. We ultimately chose co-authorship because it’s easier to interpret, and because we are focused on collaboration structure rather than influence.