we may learn that certain individuals are good at predicting the outcome of a focus group or A/B test. Peers can then predict the prediction of these individuals. This gives even faster feedback, and can free up time for that individual. Since the attention of people with accurate beliefs and good planning skills is very important, this is an important consideration.
Why do you expect this to happen?
I think you’re saying that if some masterful predictors exist, then their casting a prediction will mostly determine the outcome, sufficient for lesser people to (with a little uncertainty) learn whether they predicted well.
I suspect that for any given event, some people are better at predicting it than others. For example, in the case of UI design it is definitely the case that some people are much better than others at predicting what will do well in an A/B test. If such people are scarce, learning to predict their predictions is a good substitute for learning to predict the results of A/B tests themselves. (The real question in this situation is whether the overhead of prediction could be low enough for this to matter.)
Why do you expect this to happen?
I think you’re saying that if some masterful predictors exist, then their casting a prediction will mostly determine the outcome, sufficient for lesser people to (with a little uncertainty) learn whether they predicted well.
I suspect that for any given event, some people are better at predicting it than others. For example, in the case of UI design it is definitely the case that some people are much better than others at predicting what will do well in an A/B test. If such people are scarce, learning to predict their predictions is a good substitute for learning to predict the results of A/B tests themselves. (The real question in this situation is whether the overhead of prediction could be low enough for this to matter.)