I can’t speak for people who actually work on theoretical alignment, but my perspective is:
Yes, developing theory without the ability to empirically test your theories is really hard and does not have a good historical track record.
To do empirical work on aligning ASI, we have to build the thing that kills us, which means we die.
The seeming impossibility of theoretical alignment work isn’t a good argument that we should do empirical work instead. The two options are: we do the thing that’s really hard and probably won’t work, or we do the thing that kills us. I prefer the former.
There’s an argument you have likely encountered at least once that the empirical work on non-superintelligent alignment will be useful for aligning ASI (in Yudkowskian sense) as well, and since any human coordination is imperfect and we can only delay the development of the latter for a limited amount of time, this is the only realistic way to go.
Also, I’m pretty sure very few people in the field back at the time understood the “no historical track record” part. Seems likely to be a selection effect: the people who did probably abstained from entering AI safety in the first place
Yeah, it’s a judgment call as to whether trying to solve alignment empirically is more or less doomed than trying to coordinate to not build ASI. I don’t have a clear argument either way; the best I’ve come up with is a list of heuristic reasons why I believe the “empirical alignment” approach is more doomed, which I wrote here.
I can’t speak for people who actually work on theoretical alignment, but my perspective is:
Yes, developing theory without the ability to empirically test your theories is really hard and does not have a good historical track record.
To do empirical work on aligning ASI, we have to build the thing that kills us, which means we die.
The seeming impossibility of theoretical alignment work isn’t a good argument that we should do empirical work instead. The two options are: we do the thing that’s really hard and probably won’t work, or we do the thing that kills us. I prefer the former.
There’s an argument you have likely encountered at least once that the empirical work on non-superintelligent alignment will be useful for aligning ASI (in Yudkowskian sense) as well, and since any human coordination is imperfect and we can only delay the development of the latter for a limited amount of time, this is the only realistic way to go.
Also, I’m pretty sure very few people in the field back at the time understood the “no historical track record” part. Seems likely to be a selection effect: the people who did probably abstained from entering AI safety in the first place
Yeah, it’s a judgment call as to whether trying to solve alignment empirically is more or less doomed than trying to coordinate to not build ASI. I don’t have a clear argument either way; the best I’ve come up with is a list of heuristic reasons why I believe the “empirical alignment” approach is more doomed, which I wrote here.