Yep, I’ll bite the bullet here. This is a real problem and partly my motivation for writing the perspective explicitly.
I think people who are “in the know” are good at not over-updating on the quantitative results. And they’re good at explaining that the experiments are weak proxies which should be interpreted qualitatively at best. But people “out of the know” (e.g. junior ai safety researches) tend to overupdate and probably read the senior researchers as professing generic humility.
I would guess that even the “in the know” people are over-updating, because they usually are Not Measuring What They Think They Are Measuring even qualitatively. Like, the proxies are so weak that the hypothesis “this result will qualitatively generalize to <whatever they actually want to know about>” shouldn’t have been privileged in the first place, and the right thing for a human to do is ignore it completely.
Yep, I’ll bite the bullet here. This is a real problem and partly my motivation for writing the perspective explicitly.
I think people who are “in the know” are good at not over-updating on the quantitative results. And they’re good at explaining that the experiments are weak proxies which should be interpreted qualitatively at best. But people “out of the know” (e.g. junior ai safety researches) tend to overupdate and probably read the senior researchers as professing generic humility.
I would guess that even the “in the know” people are over-updating, because they usually are Not Measuring What They Think They Are Measuring even qualitatively. Like, the proxies are so weak that the hypothesis “this result will qualitatively generalize to <whatever they actually want to know about>” shouldn’t have been privileged in the first place, and the right thing for a human to do is ignore it completely.