If a result doesn’t generalize between optimizers, that often is a reason I’m much less interested in it, because this is evidence against it generalizing to future situations I care about. This is especially true for research that tries to learn facts, rather than research that tries to produce models with particular properties so that they can be used as subjects of research. E.g. it’s very hard to make sandbagging model organisms, and it would be sort of handy to have those model organisms around, so it is useful to know whether different optimizers help (my coworkers looked into this and didn’t make it work.)
If a result doesn’t generalize between optimizers, that often is a reason I’m much less interested in it, because this is evidence against it generalizing to future situations I care about. This is especially true for research that tries to learn facts, rather than research that tries to produce models with particular properties so that they can be used as subjects of research. E.g. it’s very hard to make sandbagging model organisms, and it would be sort of handy to have those model organisms around, so it is useful to know whether different optimizers help (my coworkers looked into this and didn’t make it work.)