adamk

• adamk 30 May 2022 20:48 UTC

  2 points

  in reply to: johnswentworth’s comment on: Un­der­stand­ing Selec­tion Theorems

  Thank you! I’d be glad to include this and any other corrections in an edit once contest results are released. Are there any other errors which catch your eye?

• adamk 26 Mar 2022 16:01 UTC

  2 points

  on: Search­ing for outliers

  I would honestly be interested to see a detailed writeup with good examples of this “maybe amazing” vs “probably good” distinction.

  A subtlety here is that the traits that make a candidate a potential outlier are often very different from the traits that would make them “pretty good,” so improving your filtering process to produce more “pretty good” candidates won’t necessarily increase the rate of finding outliers, and might even decrease it.

  Most important point I’d still want to grok is what this “might even decrease it” looks like. What are industry examples of metrics or filtering processes that can differentiate 95th percentile samples from 99.9th percentile samples? And what are some of the qualitative shifts you see between them? I suspect the art of identifying 99.9th percentile samples goes beyond looking for “really, really good” 95th-percentile-ish things.