I think you are probably unusually good at spotting which mech interp orgs are doomed ex ante, but you aren’t infallible. And I think a situation where many small startups are being founded, even if most will be doomed, is what a functional startup ecosystem looks like! We don’t want people working on obviously bad ideas, but I naively expect the process of startup ideation and experimentation, aided by VC money, to yield good mech interp directions.
I naively expect the process of startup ideation and experimentation, aided by VC money
It’s very difficult to come with AI safety startup ideas that are VC-fundable. This seems like a recipe for coming up with nice-sounding but ultimately useless ideas, or wasting a lot of effort on stuff that looks good to VCs but doesn’t advance AI safety in any way.
Maybe so! I don’t think Eric Ho’s ideas are terrible and I’ve seen for-profit AI safety startups that I like (e.g., Goodfire) and that I don’t like (e.g., Softmax, probably).
I think you are probably unusually good at spotting which mech interp orgs are doomed ex ante, but you aren’t infallible. And I think a situation where many small startups are being founded, even if most will be doomed, is what a functional startup ecosystem looks like! We don’t want people working on obviously bad ideas, but I naively expect the process of startup ideation and experimentation, aided by VC money, to yield good mech interp directions.
It’s very difficult to come with AI safety startup ideas that are VC-fundable. This seems like a recipe for coming up with nice-sounding but ultimately useless ideas, or wasting a lot of effort on stuff that looks good to VCs but doesn’t advance AI safety in any way.
Maybe so! I don’t think Eric Ho’s ideas are terrible and I’ve seen for-profit AI safety startups that I like (e.g., Goodfire) and that I don’t like (e.g., Softmax, probably).