A good point. The control group was of folks who made it through the initial screening but not the final screening, so, yes, there are differences. We explicitly discussed the possibility of randomizing admissions, but, for our first go, elected to admit the 25 people we most wanted, and to try randomizing some future events if the first worked well enough to warrant follow-ups (which it did). It is a bit of a hit to the data-gathering, but it wasn’t growth potential as such that we were selecting for—for example, younger applicants were less likely to have cool accomplishments and therefore less likely to get in, although they probably have more growth potential—so there should still be evidence in the results.
Also, we marked down which not-admitted applicants were closest to the cut-off line (there were a number of close calls; I really wished we had more than 25 spaces), so we can gain a bit of data by seeing if they were similar to the minicamp group or to the rest of the controls.
A good point. The control group was of folks who made it through the initial screening but not the final screening, so, yes, there are differences. We explicitly discussed the possibility of randomizing admissions, but, for our first go, elected to admit the 25 people we most wanted, and to try randomizing some future events if the first worked well enough to warrant follow-ups (which it did). It is a bit of a hit to the data-gathering, but it wasn’t growth potential as such that we were selecting for—for example, younger applicants were less likely to have cool accomplishments and therefore less likely to get in, although they probably have more growth potential—so there should still be evidence in the results.
Also, we marked down which not-admitted applicants were closest to the cut-off line (there were a number of close calls; I really wished we had more than 25 spaces), so we can gain a bit of data by seeing if they were similar to the minicamp group or to the rest of the controls.