MATS has steadily increased in quality over the past two years, and is now more prestigious than AISC. We also have Astra, and people who go directly to residencies at OpenAI, Anthropic, etc. One should expect that AISC doesn’t attract the best talent.
If so, AISC might not make efficient use of mentor / PI time, which is a key goal of MATS and one of the reasons it’s been successful.
AISC isn’t trying to do what MATS does. Anecdotal, but for me, MATS could not have replaced AISC (spring 2022 iteration). It’s also, as I understand it, trying to have a structure that works without established mentors, since that’s one of the large bottlenecks constraining the training pipeline.
Also, did most of the past camps ever have lots of established mentors? I thought it was just the one in 2022 that had a lot? So whatever factors made all the past AISCs work and have participants sing their praises could just still be there.
He was posting cranky technical stuff during my camp iteration too. The program was still fantastic. So whatever they are doing to make this work seems able to function despite his crankery. With a five year track record, I’m not too worried about this factor.
All but 2 of the papers listed on Manifund as coming from AISC projects are from 2021 or earlier.
In the first link at least, there are only eight papers listed in total though. With the first camp being in 2018, it doesn’t really seem like the rate dropped much? So to the extent you believe your colleagues that the camp used to be good, I don’t think the publication record is much evidence that it isn’t anymore. Paper production apparently just does not track the effectiveness of the program much. Which doesn’t surprise me, I don’t think the rate of paper producion tracks the quality of AIS research orgs much either.
The impact assessment was commissioned by AISC, not independent. They also use the number of AI alignment researchers created as an important metric. But impact is heavy-tailed, so the better metric is value of total research produced. Because there seems to be little direct research, to estimate the impact we should count the research that AISC alums from the last two years go on to produce. Unfortunately I don’t have time to do this.
Agreed on the metric being not great, and that an independently commissioned report would be better evidence (though who would have comissioned it?). But ultimately, most of what this report is apparently doing is just asking a bunch of AIS alumni what they thought of the camp and what they were up to, these days. And then noticing that these alumni often really liked it and have apparently gone on to form a significant fraction of the ecosystem. And I don’t think they even caught everyone. IIRC our AISC follow-up LTFF grant wasn’t part of the spreadsheets until I wrote Remmelt that it wasn’t there.
I am not surprised by this. Like you, my experience is that most of my current colleagues who were part of AISC tell me it was really good. The survey is just asking around and noticing the same.
I was the private donor who gave €5K. My reaction to hearing that AISC was not getting funding was that this seemed insane. The iteration I was in two years ago was fantastic for me, and the research project I got started on there is basically still continuing at Apollo now. Without AISC, I think there’s a good chance I would never have become an AI notkilleveryoneism researcher.
It feels like a very large number of people I meet in AIS today got their start in one AISC iteration or another, and many of them seem to sing its praises. I think 4⁄6 people currently on our interp team were part of one of the camps. I am not aware of any other current training program that seems to me like it would realistically replace AISC’s role, though I admittedly haven’t looked into all of them. I haven’t paid much attention to the iteration that happened in 2023, but I happen to know a bunch of people who are in the current iteration and think trying to run a training program for them is an obviously good idea.
I think MATS and co. are still way too tiny to serve all the ecosystem’s needs, and under those circumstances, shutting down a training program with an excellent five year track record seems like an even more terrible idea than usual. On top of that, the research lead structure they’ve been trying out for this camp and the last one seems to me like it might have some chance of being actually scalable. I haven’t spend much time looking at the projects for the current iteration yet, but from very brief surface exposure they didn’t seem any worse on average than the ones in my iteration. Which impressed and surprised me, because these projects were not proposed by established mentors like the ones in my iteration were. A far larger AISC wouldn’t be able to replace what a program like MATS does, but it might be able to do what AISC6 did for me, and do it for far more people than anything structured like MATS realistically ever could.
On a more meta point, I have honestly not been all that impressed with the average competency of the AIS funding ecosystem. I don’t think it not funding a project is particularly strong evidence that the project is a bad idea.
On a more meta point, I have honestly not been all that impressed with the average competency of the AIS funding ecosystem. I don’t think it not funding a project is particularly strong evidence that the project is a bad idea.
I made a different call on AISC, but also think this is right. There aren’t a lot of players in the funding ecosystem, especially post-FTX there isn’t a lot of non-OpenPhil money around, and I generally only weakly update on people succeeding to get funding or failing to get funding.
Thanks, this is pretty reassuring. Mostly due to the nonpersonal details about how AISC fits in the pipeline, but also because your work at Apollo is proprietary and so it’s not a bad sign it wasn’t published.
AISC isn’t trying to do what MATS does. Anecdotal, but for me, MATS could not have replaced AISC (spring 2022 iteration). It’s also, as I understand it, trying to have a structure that works without established mentors, since that’s one of the large bottlenecks constraining the training pipeline.
Also, did most of the past camps ever have lots of established mentors? I thought it was just the one in 2022 that had a lot? So whatever factors made all the past AISCs work and have participants sing their praises could just still be there.
He was posting cranky technical stuff during my camp iteration too. The program was still fantastic. So whatever they are doing to make this work seems able to function despite his crankery. With a five year track record, I’m not too worried about this factor.
In the first link at least, there are only eight papers listed in total though. With the first camp being in 2018, it doesn’t really seem like the rate dropped much? So to the extent you believe your colleagues that the camp used to be good, I don’t think the publication record is much evidence that it isn’t anymore. Paper production apparently just does not track the effectiveness of the program much. Which doesn’t surprise me, I don’t think the rate of paper producion tracks the quality of AIS research orgs much either.
Agreed on the metric being not great, and that an independently commissioned report would be better evidence (though who would have comissioned it?). But ultimately, most of what this report is apparently doing is just asking a bunch of AIS alumni what they thought of the camp and what they were up to, these days. And then noticing that these alumni often really liked it and have apparently gone on to form a significant fraction of the ecosystem. And I don’t think they even caught everyone. IIRC our AISC follow-up LTFF grant wasn’t part of the spreadsheets until I wrote Remmelt that it wasn’t there.
I am not surprised by this. Like you, my experience is that most of my current colleagues who were part of AISC tell me it was really good. The survey is just asking around and noticing the same.
I was the private donor who gave €5K. My reaction to hearing that AISC was not getting funding was that this seemed insane. The iteration I was in two years ago was fantastic for me, and the research project I got started on there is basically still continuing at Apollo now. Without AISC, I think there’s a good chance I would never have become an AI notkilleveryoneism researcher.
It feels like a very large number of people I meet in AIS today got their start in one AISC iteration or another, and many of them seem to sing its praises. I think 4⁄6 people currently on our interp team were part of one of the camps. I am not aware of any other current training program that seems to me like it would realistically replace AISC’s role, though I admittedly haven’t looked into all of them. I haven’t paid much attention to the iteration that happened in 2023, but I happen to know a bunch of people who are in the current iteration and think trying to run a training program for them is an obviously good idea.
I think MATS and co. are still way too tiny to serve all the ecosystem’s needs, and under those circumstances, shutting down a training program with an excellent five year track record seems like an even more terrible idea than usual. On top of that, the research lead structure they’ve been trying out for this camp and the last one seems to me like it might have some chance of being actually scalable. I haven’t spend much time looking at the projects for the current iteration yet, but from very brief surface exposure they didn’t seem any worse on average than the ones in my iteration. Which impressed and surprised me, because these projects were not proposed by established mentors like the ones in my iteration were. A far larger AISC wouldn’t be able to replace what a program like MATS does, but it might be able to do what AISC6 did for me, and do it for far more people than anything structured like MATS realistically ever could.
On a more meta point, I have honestly not been all that impressed with the average competency of the AIS funding ecosystem. I don’t think it not funding a project is particularly strong evidence that the project is a bad idea.
I made a different call on AISC, but also think this is right. There aren’t a lot of players in the funding ecosystem, especially post-FTX there isn’t a lot of non-OpenPhil money around, and I generally only weakly update on people succeeding to get funding or failing to get funding.
Thanks, this is pretty reassuring. Mostly due to the nonpersonal details about how AISC fits in the pipeline, but also because your work at Apollo is proprietary and so it’s not a bad sign it wasn’t published.