Sometimes the point is specifically to not update on the additional information, because you don’t trust yourself to update on it correctly.
Classic example: “Projects like this usually take 6 months, but looking at the plan I don’t see why it couldn’t be done in 2… wait, no, I should stick to the reference class forecast.”
Sure, but I think people often don’t do that in the best way (which is determined by what the mathematically correct way is).
Why does it make sense to use reference class forecasting in that case? Because you know you can’t trust your intuitive prior, and so you need a different starting point. But you can and should still update on the evidence you do have. If you don’t trust yourself to update correctly, that’s a much more serious problem—but make sure you’ve actually tried updating correctly first (which REQUIRES comparing how likely the evidence you see is in worlds where your prediction is true vs in worlds where its not).
I sometimes see people act like to use the “outside view” correctly, you have to just use that as your prior, and can’t update on any additional evidence you have. That is a mistake.
And the other big question with reference class forecasting is which reference class to use. And my point here is that it’s whichever reference class best summarizes your (prior) knowledge of the situation.
Sometimes the point is specifically to not update on the additional information, because you don’t trust yourself to update on it correctly.
Classic example: “Projects like this usually take 6 months, but looking at the plan I don’t see why it couldn’t be done in 2… wait, no, I should stick to the reference class forecast.”
Sure, but I think people often don’t do that in the best way (which is determined by what the mathematically correct way is).
Why does it make sense to use reference class forecasting in that case? Because you know you can’t trust your intuitive prior, and so you need a different starting point. But you can and should still update on the evidence you do have. If you don’t trust yourself to update correctly, that’s a much more serious problem—but make sure you’ve actually tried updating correctly first (which REQUIRES comparing how likely the evidence you see is in worlds where your prediction is true vs in worlds where its not).
I sometimes see people act like to use the “outside view” correctly, you have to just use that as your prior, and can’t update on any additional evidence you have. That is a mistake.
And the other big question with reference class forecasting is which reference class to use. And my point here is that it’s whichever reference class best summarizes your (prior) knowledge of the situation.