If you were to be less charitable than I would, this would explain a lot about why AI safety wants to regulate AI companies so much, since they’re offering at least a partial solution, if not a full solution to the alignment problem and safety problem that doesn’t require much slowdown in AI progress, nor does it require donations to MIRI or classic AI safety organizations, nor does it require much coordination, which threatens both AI safety funding sources and fears that their preferred solution, slowing down AI won’t be implemented.
I think this is kind of a non-sequitur and also wrong in multiple ways. Slowdown can give more time either for work like Davidad’s or improvements to RLHF-like techniques. Most of the AI safety people I know have actual models of why RLHF will stop working based on reasonable assumptions.
A basic fact about EA is that it’s super consequentialist and thus less susceptible to this “personal sacrifice = good” mistake than most other groups, and the AI alignment researchers who are not EAs are just normal ML researchers. Just look at the focus on cage-free campaigns over veganism, or earning-to-give. Not saying it’s impossible for AI safety researchers to make this mistake, but you have no reason to believe they are.
I think this is kind of a non-sequitur and also wrong in multiple ways. Slowdown can give more time either for work like Davidad’s or improvements to RLHF-like techniques. Most of the AI safety people I know have actual models of why RLHF will stop working based on reasonable assumptions.
A basic fact about EA is that it’s super consequentialist and thus less susceptible to this “personal sacrifice = good” mistake than most other groups, and the AI alignment researchers who are not EAs are just normal ML researchers. Just look at the focus on cage-free campaigns over veganism, or earning-to-give. Not saying it’s impossible for AI safety researchers to make this mistake, but you have no reason to believe they are.