Thanks for asking. I do indeed think that setup could be a very bad idea. You train for agency, you might well get agency, and that agency might be broadly scoped.
(It’s still not obvious to me that that setup leads to doom by default, though. Just more dangerous than pretraining LLMs.)
Thanks for asking. I do indeed think that setup could be a very bad idea. You train for agency, you might well get agency, and that agency might be broadly scoped.
(It’s still not obvious to me that that setup leads to doom by default, though. Just more dangerous than pretraining LLMs.)