I’ve been doing some EDA on these datasets, vibe-coded this plot with confidence intervals, and remembered this discussion, so I figured why not drop it here:
Good catch! I think maybe it’s because I’m looking at data ~5 months newer from stronger models they’ve run, hence the “average success rate” of the orange dots has crept up a bit? At least, the x-values of all points look consistent to me e.g. the left 6 points.
I’ve been doing some EDA on these datasets, vibe-coded this plot with confidence intervals, and remembered this discussion, so I figured why not drop it here:
These points do not seem like the same points as the other plot? (Look at the x-y of the orange dots).
Good catch! I think maybe it’s because I’m looking at data ~5 months newer from stronger models they’ve run, hence the “average success rate” of the orange dots has crept up a bit? At least, the x-values of all points look consistent to me e.g. the left 6 points.