Ah yes. I actually missed that you’d scoped that statement to general purpose software engineering. That is indeed one of the most relevant capabilities. I was thinking of general purpose problem-solving, another of the most critical capabilities for AI to become really dangerous.
I agree that even if scaffolding could work, RL on long CoT does something similar, and that’s where the effort and momentum is going.
AIs writing and testing scaffolds semi-autonomously is something I hadn’t considered. There might be a pretty tight loop that could make that effective.
Ah yes. I actually missed that you’d scoped that statement to general purpose software engineering. That is indeed one of the most relevant capabilities. I was thinking of general purpose problem-solving, another of the most critical capabilities for AI to become really dangerous.
I agree that even if scaffolding could work, RL on long CoT does something similar, and that’s where the effort and momentum is going.
AIs writing and testing scaffolds semi-autonomously is something I hadn’t considered. There might be a pretty tight loop that could make that effective.