A key crux is I don’t generally agree with this claim in AI safety:
A thinks the detail really won’t change the conclusion, and thinks this just misses the point, but doesn’t want to spend time, because getting all the details exactly right would take maybe a decade.
In this specific instance, it could work, but in general I think ignoring details is a core failure mode of people that tend towards abstract/meta stuff, which is absolutely the case on Lesswrong.
I think abstraction/meta/theoretical work is useful, but also that theory absolutely does require empirics to make sure you are focusing on the relevant parts of the problem.
This especially is the case if you are focused on working on solutions, rather than trying to get attention on a problem.
I’ll just quote from Richard Ngo here, because he made the point shorter than I can (it’s in a specific setting, but the general point holds):
I currently think of Eliezer as someone who has excellent intuitions about the broad direction of progress at a very high level of abstraction—but where the very fact that these intuitions are so abstract rules out the types of path-dependencies that I expect solutions to alignment will actually rely on. At this point, people who find Eliezer’s intuitions compelling should probably focus on fleshing them out in detail—e.g. using toy models, or trying to decompose the concept of consequentialism—rather than defending them at a high level.
But the problem is that we likely don’t have time to flesh out all the details or do all the relevant experiments before it might be too late, and governments need to understand that based on arguments that therefore cannot possibly rely on everything being fleshed out.
Of course I want people to gather as much important empirical evidence and concrete detailed theory as possible asap.
Also, the pre-everything-worked-out-in-detail arguments also need to inform which experiments are done, and so that is why people who have actually listened to those pre-detailed arguments end up on average doing much more relevant empirical work IMO.
A key crux is I don’t generally agree with this claim in AI safety:
In this specific instance, it could work, but in general I think ignoring details is a core failure mode of people that tend towards abstract/meta stuff, which is absolutely the case on Lesswrong.
I think abstraction/meta/theoretical work is useful, but also that theory absolutely does require empirics to make sure you are focusing on the relevant parts of the problem.
This especially is the case if you are focused on working on solutions, rather than trying to get attention on a problem.
I’ll just quote from Richard Ngo here, because he made the point shorter than I can (it’s in a specific setting, but the general point holds):
But the problem is that we likely don’t have time to flesh out all the details or do all the relevant experiments before it might be too late, and governments need to understand that based on arguments that therefore cannot possibly rely on everything being fleshed out.
Of course I want people to gather as much important empirical evidence and concrete detailed theory as possible asap.
Also, the pre-everything-worked-out-in-detail arguments also need to inform which experiments are done, and so that is why people who have actually listened to those pre-detailed arguments end up on average doing much more relevant empirical work IMO.