I would expect the bigger risk to not be dropping either set of constraints, but generalizing to “Don’t do things society as of 2026 thinks are bad” instead of “Don’t do things that are bad under some reasonable set of moral foundations”.
I would expect the bigger risk to not be dropping either set of constraints, but generalizing to “Don’t do things society as of 2026 thinks are bad” instead of “Don’t do things that are bad under some reasonable set of moral foundations”.