This was a fantastic read. Among my top three (at least) on Goodhart!
Stupid simple observation: if you could get enough independent[1] evaluations of X you could smooth out heavy tails by ensembling (by central limit theorem).
actually independent, not like asking lots of humans to ‘independently’ rate something, which is obviously correlated in important ways—I think this condition is very hard to achieve in reality
This was a fantastic read. Among my top three (at least) on Goodhart!
Stupid simple observation: if you could get enough independent[1] evaluations of X you could smooth out heavy tails by ensembling (by central limit theorem).
actually independent, not like asking lots of humans to ‘independently’ rate something, which is obviously correlated in important ways—I think this condition is very hard to achieve in reality