This question helped me realize that, if we have a theory that retrospective forecasting works, we can use that theory to make testable predictions, and then we can build up evidence for or against retrospective forecasting.
Suppose we have two models, Model-A and Model-B. The prior is 50%. We also have a history to look at. We can apply retrospective forecasting to determine P(History|ModelA) and P(History|ModelB), and then from Bayes Theorem we can update our estimate as to which model is most likely. Suppose this tells us that Model-A is much more likely.
Now we can use ModelA and ModelB to make testable predictions about the future. As the future unfolds, if events occur as predicted by ModelA, this is evidence that retrospective forecasting works. If events occur as predicted by ModelB, this is evidence that retrospective forecasting doesn’t work.
This question helped me realize that, if we have a theory that retrospective forecasting works, we can use that theory to make testable predictions, and then we can build up evidence for or against retrospective forecasting.
Suppose we have two models, Model-A and Model-B. The prior is 50%. We also have a history to look at. We can apply retrospective forecasting to determine P(History|ModelA) and P(History|ModelB), and then from Bayes Theorem we can update our estimate as to which model is most likely. Suppose this tells us that Model-A is much more likely.
Now we can use ModelA and ModelB to make testable predictions about the future. As the future unfolds, if events occur as predicted by ModelA, this is evidence that retrospective forecasting works. If events occur as predicted by ModelB, this is evidence that retrospective forecasting doesn’t work.