Thanks for posting this! I have a longer reply to Taleb’s post that I’ll post soon. But first:
When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with? Or do you do a mental adjustment or discounting based on some factor you think they’ve left out?
I think it depends on the model. First, note that all forecasting models only take into account a specific set of signals. If there are factors influencing the vote that I’m both aware of and don’t think are reflected in the signals, then you should update their forecast to reflect this. For example, I think that because Nate Silver’s model was based on polls that lag behind current events, if you had some evidence that a given event was really bad or really good for one of the two candidates, such as the Comey letter or the Trump video, you should update in favor of/against a Trump Presidency before it becomes reflected in the polls.
The math is based on assumptions though that with high uncertainty, far out from the election, the best forecast is 50-50.
Not really. The key assumption is that your forecasts are a Wiener process—a continuous time martingale with normally-distributed increments. (I find this funny because Taleb spends multiplebooks railing against normality assumptions.) This is kind of a troubling assumption, as Lumifer points out below. If your forecast is continuous (though it need not be), then it can be thought of as a time-transformed Wiener process, but as far as I can tell he doesn’t account for the time-transformation.
Everyone agrees that as uncertainty becomes really high, the best forecast is 50-50. Conversely, if you make a confident forecast (say 90-10) and you’re properly calibrated, you’re also implying that you’re unlikely to change your forecast by very much in the future (with high probability, you won’t forecast 1-99).
I think the question to ask is—how much volatility should make you doubt a forecast? If someone’s forecast varied daily between 1-99 and 99-1, you might learn to just ignore them, for example. Taleb tries to offer one answer to this, but makes some questionable assumptions along the way and I don’t really agree with his result.
Thanks for posting this! I have a longer reply to Taleb’s post that I’ll post soon. But first:
I think it depends on the model. First, note that all forecasting models only take into account a specific set of signals. If there are factors influencing the vote that I’m both aware of and don’t think are reflected in the signals, then you should update their forecast to reflect this. For example, I think that because Nate Silver’s model was based on polls that lag behind current events, if you had some evidence that a given event was really bad or really good for one of the two candidates, such as the Comey letter or the Trump video, you should update in favor of/against a Trump Presidency before it becomes reflected in the polls.
Not really. The key assumption is that your forecasts are a Wiener process—a continuous time martingale with normally-distributed increments. (I find this funny because Taleb spends multiple books railing against normality assumptions.) This is kind of a troubling assumption, as Lumifer points out below. If your forecast is continuous (though it need not be), then it can be thought of as a time-transformed Wiener process, but as far as I can tell he doesn’t account for the time-transformation.
Everyone agrees that as uncertainty becomes really high, the best forecast is 50-50. Conversely, if you make a confident forecast (say 90-10) and you’re properly calibrated, you’re also implying that you’re unlikely to change your forecast by very much in the future (with high probability, you won’t forecast 1-99).
I think the question to ask is—how much volatility should make you doubt a forecast? If someone’s forecast varied daily between 1-99 and 99-1, you might learn to just ignore them, for example. Taleb tries to offer one answer to this, but makes some questionable assumptions along the way and I don’t really agree with his result.