This really resonated with me. I am a student doing alignment research on a pretty nontraditional path. I work in mental health, and alongside that I run experiments on my own local hardware. I recently published my first paper on sparse autoencoder analysis of Anthropic’s deceptive AI model organism. I do not have prestigious affiliations or a PhD, just a lot of curiosity and a willingness to put in the hours.
Your diagnosis feels right. The real bottleneck does not seem to be talent, but the lack of infrastructure to find it and support it. When people who are genuinely capable are being turned away at rates above ninety eight percent, that starts to look less like selectivity and more like a coordination failure.
The bounty idea appeals to me because it sidesteps credential based gatekeeping. You do not need permission from an institution to try to do the work. You just do it, and the results speak for themselves. That said, I am less convinced by the prediction market style of verification. Citation counts tend to reward people who are already embedded in academic networks, and waiting a year for feedback is especially hard when you are trying to build momentum.
One piece I did not see discussed is the role AI tools themselves can play in easing the mentorship bottleneck. In my own work, collaborating with capable AI systems has helped fill some of the gaps you would normally expect a mentor to cover, especially around fast feedback, synthesizing literature, and iterating on experimental ideas. It is not a replacement for real mentorship, but it feels like a meaningful third path alongside bounties and expanded fellowship programs.
Thanks for writing this. It is an important conversation, and I am glad to see it happening!
I don’t like to self-publicize, but I think you’d really resonate with a piece I wrote a while back, it went semi-viral and resulted in some very interesting discussion. It’s about the systematic biases that expertise invokes, and what that’s like as a novice: https://boydkane.com/essays/experts
Do not even worry about self-publicizing. I am willing to read anything interesting! Also sorry for the delay, been deep in experiments/work/school/holiday stuff. I’ll give it a read right now though. The novice/expert dynamic is definitely something I think about with alignment research.
I am less convinced by the prediction market style of verification
I’m also not super convinced, but I do think the problem of verifying solutions is a big one, so I wanted to put out some alternate answer out there.
the role AI tools themselves can play in easing the mentorship bottleneck
For guiding up-and-coming researchers I definitely agree that existing AIs can help, although I also feel that each person should find something that works for them.
For using AIs to review submissions, I’m not sure the AIs are good enough yet to do a full review, but maybe they can significantly reduce the number of low-effort papers that a researcher has to review. E.g. use an LLM to check for typos, style, prior work, whether the paper actually answers the question, etc.
This really resonated with me. I am a student doing alignment research on a pretty nontraditional path. I work in mental health, and alongside that I run experiments on my own local hardware. I recently published my first paper on sparse autoencoder analysis of Anthropic’s deceptive AI model organism. I do not have prestigious affiliations or a PhD, just a lot of curiosity and a willingness to put in the hours.
Your diagnosis feels right. The real bottleneck does not seem to be talent, but the lack of infrastructure to find it and support it. When people who are genuinely capable are being turned away at rates above ninety eight percent, that starts to look less like selectivity and more like a coordination failure.
The bounty idea appeals to me because it sidesteps credential based gatekeeping. You do not need permission from an institution to try to do the work. You just do it, and the results speak for themselves. That said, I am less convinced by the prediction market style of verification. Citation counts tend to reward people who are already embedded in academic networks, and waiting a year for feedback is especially hard when you are trying to build momentum.
One piece I did not see discussed is the role AI tools themselves can play in easing the mentorship bottleneck. In my own work, collaborating with capable AI systems has helped fill some of the gaps you would normally expect a mentor to cover, especially around fast feedback, synthesizing literature, and iterating on experimental ideas. It is not a replacement for real mentorship, but it feels like a meaningful third path alongside bounties and expanded fellowship programs.
Thanks for writing this. It is an important conversation, and I am glad to see it happening!
I don’t like to self-publicize, but I think you’d really resonate with a piece I wrote a while back, it went semi-viral and resulted in some very interesting discussion. It’s about the systematic biases that expertise invokes, and what that’s like as a novice: https://boydkane.com/essays/experts
Do not even worry about self-publicizing. I am willing to read anything interesting! Also sorry for the delay, been deep in experiments/work/school/holiday stuff. I’ll give it a read right now though. The novice/expert dynamic is definitely something I think about with alignment research.
I’m also not super convinced, but I do think the problem of verifying solutions is a big one, so I wanted to put out some alternate answer out there.
For guiding up-and-coming researchers I definitely agree that existing AIs can help, although I also feel that each person should find something that works for them.
For using AIs to review submissions, I’m not sure the AIs are good enough yet to do a full review, but maybe they can significantly reduce the number of low-effort papers that a researcher has to review. E.g. use an LLM to check for typos, style, prior work, whether the paper actually answers the question, etc.