[Question] How good a proxy for accuracy is precision?



Ac­cu­racy = av­er­ag­ing to the right an­swer.

Pre­ci­sion = hav­ing a low stan­dard de­vi­a­tion.


It’s easy to get a pre­cise but in­ac­cu­rate an­swer- just guess 0 ev­ery time. But for situ­a­tions where peo­ple aren’t good­hart­ing, are a cluster of an­swers that are more similar to each other more likely to av­er­age to the cor­rect an­swer than a cluster of an­swers that aren’t? And how does that change based on how the an­swers were gen­er­ated (e.g. same in­di­vi­d­ual giv­ing mul­ti­ple an­swers over time vs. mul­ti­ple in­di­vi­d­u­als an­swer­ing vs. mul­ti­ple groups an­swer­ing)? In what ways do ac­cu­racy and pre­ci­sion di­rectly trade off against each other?