Being ~50% of where people were thinking about AI alignment until about 2018 - putting out educational materials, running workshops and conferences, etc. Each individual thing is fairly small, but they add up.
Publishing basic explainers in respectable enough formats that academics have citations for them (especially Soares and Fallenstein 2014).
Jessica’s Quantilizers paper.
Evan’s Risks From Learned Optimization.
Peter de Blanc’s Ontological Crises paper.
Eliezer’s Intelligence Explosion Microeconomics and related arguments.
(Probably some other publications I’m forgetting)
Blue-sky research on doing new things that might be good (probably non-disclosed stuff, infrabayesianism might go here or below).
All the stuff that’s interesting and was probably was good for the field, but turned out not super relevant to training big neural nets, but still might turn out to be useful to have in our toolbox (decision theory, logical inductors, open source code games, etc.)
Being ~50% of where people were thinking about AI alignment until about 2018 - putting out educational materials, running workshops and conferences, etc.
I think this is important to mention- from 2000 to 2018 they were doing basically all the heavy lifting, and 2018-2022 was a low period of contributions. That’s a pretty great ratio of peak to valley.
They also spent almost all of that second period trying to find a way out by coming across something big again, like they’d been for almost two years prior; their work with CFAR seems to me to have been a solid bet at the time (in fact I myself am still betting on CFAR in 2023, in spite of everything).
Being ~50% of where people were thinking about AI alignment until about 2018 - putting out educational materials, running workshops and conferences, etc. Each individual thing is fairly small, but they add up.
Publishing basic explainers in respectable enough formats that academics have citations for them (especially Soares and Fallenstein 2014).
Jessica’s Quantilizers paper.
Evan’s Risks From Learned Optimization.
Peter de Blanc’s Ontological Crises paper.
Eliezer’s Intelligence Explosion Microeconomics and related arguments.
(Probably some other publications I’m forgetting)
Blue-sky research on doing new things that might be good (probably non-disclosed stuff, infrabayesianism might go here or below).
All the stuff that’s interesting and was probably was good for the field, but turned out not super relevant to training big neural nets, but still might turn out to be useful to have in our toolbox (decision theory, logical inductors, open source code games, etc.)
I think this is important to mention- from 2000 to 2018 they were doing basically all the heavy lifting, and 2018-2022 was a low period of contributions. That’s a pretty great ratio of peak to valley.
They also spent almost all of that second period trying to find a way out by coming across something big again, like they’d been for almost two years prior; their work with CFAR seems to me to have been a solid bet at the time (in fact I myself am still betting on CFAR in 2023, in spite of everything).
Can you point me to the work they did with CFAR?
I agree that the work on ontological crises was good, and feels like a strong precursor to model-splintering and concept/value extrapolation.