One of the reasons for differential advances in many areas of automated prosaic AI safety research should also be pretty intuitive: they tend to require a lot less compute; so the automated researchers can make a lot more attempts at the problem per compute budget.
(Those are not central examples of the kind of work necessary to align superintelligent AI systems. We can rehash that whole argument here, but that seems unlikely to get anywhere.)
Disagree. I don’t think the object-level claim is obvious even for near-term, same-paradigm systems, and in any case, there are some ways to bootstrap the work of current x-safe automated systems to the automation of harder-to-evaluate work, through e.g. weak-to-strong research as mentioned in ‘Automated Weak-to-Strong Research’, or through automating reviewing, or through automating tasks from the WBE workflow (like image segmentation and proofreading). I might write more about this later in a separate shortform/post.
Strong disagree. AFAICT, there have already been stronger published results for automated AI safety research than for automated AI capabilities. E.g. I’m unaware of any comparably strong capabilities-relevant results as in Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLM or in Automated Weak-to-Strong Researcher, and I spend large amounts of time engaging with both of these kinds of research.
One of the reasons for differential advances in many areas of automated prosaic AI safety research should also be pretty intuitive: they tend to require a lot less compute; so the automated researchers can make a lot more attempts at the problem per compute budget.
(Those are not central examples of the kind of work necessary to align superintelligent AI systems. We can rehash that whole argument here, but that seems unlikely to get anywhere.)
Disagree. I don’t think the object-level claim is obvious even for near-term, same-paradigm systems, and in any case, there are some ways to bootstrap the work of current x-safe automated systems to the automation of harder-to-evaluate work, through e.g. weak-to-strong research as mentioned in ‘Automated Weak-to-Strong Research’, or through automating reviewing, or through automating tasks from the WBE workflow (like image segmentation and proofreading). I might write more about this later in a separate shortform/post.