First, I want to dispute the statement that a 50% is uninformative. It can be very informative depending on value of the outcomes. E.g., if I am analyzing transactions looking for fraud, that a transaction has 50% prediction of being fraudulent is “very informative”: most fraudulent transactions may have fraud probabilities much, much lower than that.
Second, it is true that beliefs on probabilities need not be “sharp”. The Bayesian approach to the problem (which is in fact the very problem that Bayes originally discussed!) would require you to provide a distribution of your “expected” (I want to avoid the terms “prior” or “subjective” explicitly here) probabilities. Such distribution could be more or less concentrated. The beta distribution could be used to encode such uncertainty; actually, it is the canonical distribution to do so. The question would remain how to operationalize it in a prediction market, particularly from the UX point of view.
First, I want to dispute the statement that a 50% is uninformative. It can be very informative depending on value of the outcomes.
Yes, absolutely. 50% can be incredibly useful. Unfortunately, it also represents the “I don’t know” calibration option in most prediction markets. A market at 50% for “Will we discover a civilization ending asteroid in the next 50 years?” would be cause for much concern.
Is the market really saying that discovering this asteroid is essentially a coin flip with 1:1 odds? More likely it just represents the entire market saying “I don’t know”. It’s these types of 50% that are considered useless, but I think do still convey information—especially if saying “I don’t know” is an informed opinion.
The Bayesian approach to the problem (which is in fact the very problem that Bayes originally discussed!) would require you to provide a distribution of your “expected” (I want to avoid the terms “prior” or “subjective” explicitly here) probabilities
I think there might an ontological misunderstanding here? I fully agree that ones expectations are often best represented by a non-normal distribution of outcomes. But this presumes that such a distribution “exists”? If it does, then one way to capture it would be to place multiple bets at different levels like one does with options for a stock. Metaculus already captures this distribution for the market as a whole—but only for those who were confident and certain enough to place bets.
My suggestion is to also capture signal from those with studied uncertainty who don’t feel comfortable placing bets on ANY distribution. It’s not that their distribution is flat—it’s that for them a meaningful distribution does not exist. Their belief is “doubt that a meaningful prediction is even possible”.
First, I want to dispute the statement that a 50% is uninformative. It can be very informative depending on value of the outcomes. E.g., if I am analyzing transactions looking for fraud, that a transaction has 50% prediction of being fraudulent is “very informative”: most fraudulent transactions may have fraud probabilities much, much lower than that.
Second, it is true that beliefs on probabilities need not be “sharp”. The Bayesian approach to the problem (which is in fact the very problem that Bayes originally discussed!) would require you to provide a distribution of your “expected” (I want to avoid the terms “prior” or “subjective” explicitly here) probabilities. Such distribution could be more or less concentrated. The beta distribution could be used to encode such uncertainty; actually, it is the canonical distribution to do so. The question would remain how to operationalize it in a prediction market, particularly from the UX point of view.
Yes, absolutely. 50% can be incredibly useful. Unfortunately, it also represents the “I don’t know” calibration option in most prediction markets. A market at 50% for “Will we discover a civilization ending asteroid in the next 50 years?” would be cause for much concern.
Is the market really saying that discovering this asteroid is essentially a coin flip with 1:1 odds? More likely it just represents the entire market saying “I don’t know”. It’s these types of 50% that are considered useless, but I think do still convey information—especially if saying “I don’t know” is an informed opinion.
I think there might an ontological misunderstanding here? I fully agree that ones expectations are often best represented by a non-normal distribution of outcomes. But this presumes that such a distribution “exists”? If it does, then one way to capture it would be to place multiple bets at different levels like one does with options for a stock. Metaculus already captures this distribution for the market as a whole—but only for those who were confident and certain enough to place bets.
My suggestion is to also capture signal from those with studied uncertainty who don’t feel comfortable placing bets on ANY distribution. It’s not that their distribution is flat—it’s that for them a meaningful distribution does not exist. Their belief is “doubt that a meaningful prediction is even possible”.