Not for the first time, a draft paper by Nassim Taleb makes me think, “mathematically, the paper looks fine, and congratulations on knowing calculus, but isn’t this waffling past the issue people are actually interested in?”
Many people who trusted poll-based forecasts were indeed shocked on election day, but not so much because the months-long sequence of forecasts wobbled back & forth too much. They were shocked more because the final forecasts failed to predict the final outcome. And the problem there was the difficulty of polling representative samples of people who’d actually vote, a very different problem to the one Taleb identifies.
I also have a specific question to tie this back to a rationality based framework: When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with? Or do you do a mental adjustment or discounting based on some factor you think they’ve left out?
In the specific case of the US election, I did use Silver’s forecasts as a guideline, but I also considered a model based on fundamentals (income growth and dead American soldiers), not polls: Doug Hibbs’s Bread & Peace model. Hibbs predicted a 53%-54% split in Clinton’s favour of the Dem/Rep vote. (That 53%-54% prediction might sound pretty poor with hindsight, but the expected error attached to the prediction was 2%, and Clinton has 51%, so it’s within statistical bounds.)
Whether it’s prediction market variations
I didn’t really pay attention to prediction markets. For elections & referendums I don’t believe prediction markets add anything significant beyond polls & fundamentals.
or adjustments based on perceiving changes in nationalism or politician specific skills
I didn’t consciously adjust on that basis. As far as I know, ordinary, retrospective economic voting played a big role in explaining even “the Extraordinary Election of Adolf Hitler”, so I figured I wouldn’t bother putting my thumb on the scale because of vigorous nationalism or charisma.
Upvoted your post as encouragement to post more stuff like this. I shrugged my shoulders at Taleb’s paper, but the topic’s fascinating and I’d like to see more stuff like Taleb’s paper, even if the paper itself is unimpressive.
This comment brought to you by sarahconstantin’s and AnnaSalamon’s Your Wannabe Rationalist Community Blog Needs You! posts. Without them I’d likely have carried on my 11-month streak of not posting.
In a lot of cases, yes, although some of the shocked are presumably guilty merely of e.g. following Sam Wang’s forecasts (which gave Clinton 50:1 or better odds) rather than Silver’s.
Not for the first time, a draft paper by Nassim Taleb makes me think, “mathematically, the paper looks fine, and congratulations on knowing calculus, but isn’t this waffling past the issue people are actually interested in?”
Many people who trusted poll-based forecasts were indeed shocked on election day, but not so much because the months-long sequence of forecasts wobbled back & forth too much. They were shocked more because the final forecasts failed to predict the final outcome. And the problem there was the difficulty of polling representative samples of people who’d actually vote, a very different problem to the one Taleb identifies.
In the specific case of the US election, I did use Silver’s forecasts as a guideline, but I also considered a model based on fundamentals (income growth and dead American soldiers), not polls: Doug Hibbs’s Bread & Peace model. Hibbs predicted a 53%-54% split in Clinton’s favour of the Dem/Rep vote. (That 53%-54% prediction might sound pretty poor with hindsight, but the expected error attached to the prediction was 2%, and Clinton has 51%, so it’s within statistical bounds.)
I didn’t really pay attention to prediction markets. For elections & referendums I don’t believe prediction markets add anything significant beyond polls & fundamentals.
I didn’t consciously adjust on that basis. As far as I know, ordinary, retrospective economic voting played a big role in explaining even “the Extraordinary Election of Adolf Hitler”, so I figured I wouldn’t bother putting my thumb on the scale because of vigorous nationalism or charisma.
Upvoted your post as encouragement to post more stuff like this. I shrugged my shoulders at Taleb’s paper, but the topic’s fascinating and I’d like to see more stuff like Taleb’s paper, even if the paper itself is unimpressive.
This comment brought to you by sarahconstantin’s and AnnaSalamon’s Your Wannabe Rationalist Community Blog Needs You! posts. Without them I’d likely have carried on my 11-month streak of not posting.
Those are the people who don’t understand the difference between a point forecast and a distribution forecast.
In a lot of cases, yes, although some of the shocked are presumably guilty merely of e.g. following Sam Wang’s forecasts (which gave Clinton 50:1 or better odds) rather than Silver’s.