WIth Eisenhower, I think planning is vital work. And I think the field of AGI safety is doing far too little of it.
This is standard in science, and alignment is largely a science. But in most areas of science it’s okay to make slow and inefficient progress, and to make lots of mistakes.
In alignment, those things are definitely not okay. We need to be more efficient than any previous scientific effort (except perhaps the human genome project—an interesting model but I’ll leave that aside for now).
That means making plans, primarily as a forcing function for thinking realistically about the realities of the challenge we’re facing, and the most likely specific scenarios that might change that challenge in important ways.
The responses to @Marius Hobbhahn’sWhat’s the short timeline plan? convinced me that we are desperately in need of better plans for alignment. Most of the plans listed there were simply not about alignment, but about control and interpretability. Those are helpful for alignment, but not in and of themselves a plan.
The plan “we’ll figure it out when we get there; in the meantime we’ll buy time” is not a good plan. It’s the realistic basis, but more planning is clear possible and I think almost certainly useful.
Some people argue that there are already too many plans. That is because they are bad plans that don’t accomplish the purpose of planning as forcing careful thought about the situation you are actually in.
Plans are worthless, but planning is everything.
WIth Eisenhower, I think planning is vital work. And I think the field of AGI safety is doing far too little of it.
This is standard in science, and alignment is largely a science. But in most areas of science it’s okay to make slow and inefficient progress, and to make lots of mistakes.
In alignment, those things are definitely not okay. We need to be more efficient than any previous scientific effort (except perhaps the human genome project—an interesting model but I’ll leave that aside for now).
That means making plans, primarily as a forcing function for thinking realistically about the realities of the challenge we’re facing, and the most likely specific scenarios that might change that challenge in important ways.
The responses to @Marius Hobbhahn’s What’s the short timeline plan? convinced me that we are desperately in need of better plans for alignment. Most of the plans listed there were simply not about alignment, but about control and interpretability. Those are helpful for alignment, but not in and of themselves a plan.
The plan “we’ll figure it out when we get there; in the meantime we’ll buy time” is not a good plan. It’s the realistic basis, but more planning is clear possible and I think almost certainly useful.
Some people argue that there are already too many plans. That is because they are bad plans that don’t accomplish the purpose of planning as forcing careful thought about the situation you are actually in.