It’s not a crux in my view. I think the choice of method (LLM/modified-from-LLM/non-LLM) has some impact on the timeline distribution, but not on policy or alignment research.
Trying to make an intervention narrow enough that it only covers one architecture or research area probably won’t succeed to cover even that much, and we probably need to worry about multiple risk vectors.
I believe the kind of alignment research that is based on LLMs is net-negative, since it safety-washes a fundamentally dangerous and morally bankrupt pursuit of superintelligence. We know prosaic alignment can’t work in the limit, but it can look good to would-be regulators.
Policy has to address all possible vectors of creating superintelligence so that none can be legally or covertly pursued. Just calling it “general-purpose AI” (or similar) has been sufficient for now to convince governments to get involved. That said, LLM demos are sufficient as examples to get most of the point across, without needing to get into any technical details (beyond “there are probably a few different ways to make really powerful AI”). Preventing giant training runs is important, and in the long term (we don’t know how soon), we’ll need some classes of theoretical AI research to be treated like CSAM online (but with better enforcement than has been achieved with that to date).
It’s not a crux in my view. I think the choice of method (LLM/modified-from-LLM/non-LLM) has some impact on the timeline distribution, but not on policy or alignment research.
Trying to make an intervention narrow enough that it only covers one architecture or research area probably won’t succeed to cover even that much, and we probably need to worry about multiple risk vectors.
I believe the kind of alignment research that is based on LLMs is net-negative, since it safety-washes a fundamentally dangerous and morally bankrupt pursuit of superintelligence. We know prosaic alignment can’t work in the limit, but it can look good to would-be regulators.
Policy has to address all possible vectors of creating superintelligence so that none can be legally or covertly pursued. Just calling it “general-purpose AI” (or similar) has been sufficient for now to convince governments to get involved. That said, LLM demos are sufficient as examples to get most of the point across, without needing to get into any technical details (beyond “there are probably a few different ways to make really powerful AI”). Preventing giant training runs is important, and in the long term (we don’t know how soon), we’ll need some classes of theoretical AI research to be treated like CSAM online (but with better enforcement than has been achieved with that to date).