Manifold has calibration statistics about their markets. They seem well calibrated and the accuracy peaks around 10-20 traders. They even cite a paper from 2007 looking at small markets in 2006 that concludes that 16 traders should be sufficient. Polymarket plots brier score vs volume with .09 up to $10k, then .08 up to $25k, .07 up to $100k, .05 up to $250k, .04 up to $500k, .03 up to $1M, .02 for $1M+. Brier score is mean squared error (e.g. if I say 70% and it happens I have a score of .09), so that suggests it’s fairly good even at relatively low liquidities.
The way I see it, it being cheap to swing the market isn’t sufficient for manipulation. The question is how fast it gets corrected by other traders. Basically, if someone (or someone’s trading bot) notices they can make bank. The theory then goes that manipulators increase accuracy by incentivizing informed trading—though I think this is a long run thing.
I tested this by manipulating a scandal market about Manifold CEO Austin Chen, with his permission. It was originally at 4%; at 4:50 PM California time, I spent M$200 in play money (=~ $2 in real money) to manipulate it up to 95%. By 5:30 California time, it was back down to 4%. This isn’t a great example, because real attackers might be more subtle, make multiple small bets, only try to push it a few percent, etc—but I think it’s a pretty good sign.
(the embed says the market currently has 78 traders and M48k liquidity, which is pretty high for Manifold)
It also cites an old paper by Robin Hanson, which in the introduction cites a few other papers showing failed manipulation in both the field and lab experiments. I don’t know of bigger attempts now that prediction markets are mainstream, and would be interested in seeing data about Polymarket and Predictit.
Prediction markets of course have problems, but they seem to not get much worse at mid-low liquidity.
I think you are correct to look at activity though. Will activity will spike back up after manipulation? Perhaps someone should try noise trading a couple inactive markets as an experiment.
Manifold has calibration statistics about their markets. They seem well calibrated and the accuracy peaks around 10-20 traders. They even cite a paper from 2007 looking at small markets in 2006 that concludes that 16 traders should be sufficient. Polymarket plots brier score vs volume with .09 up to $10k, then .08 up to $25k, .07 up to $100k, .05 up to $250k, .04 up to $500k, .03 up to $1M, .02 for $1M+. Brier score is mean squared error (e.g. if I say 70% and it happens I have a score of .09), so that suggests it’s fairly good even at relatively low liquidities.
The way I see it, it being cheap to swing the market isn’t sufficient for manipulation. The question is how fast it gets corrected by other traders. Basically, if someone (or someone’s trading bot) notices they can make bank. The theory then goes that manipulators increase accuracy by incentivizing informed trading—though I think this is a long run thing.
On manipulation, Manifold cites Scott Alexander’s manipulation experiment, which got corrected in an hour:
It also cites an old paper by Robin Hanson, which in the introduction cites a few other papers showing failed manipulation in both the field and lab experiments. I don’t know of bigger attempts now that prediction markets are mainstream, and would be interested in seeing data about Polymarket and Predictit.
Prediction markets of course have problems, but they seem to not get much worse at mid-low liquidity.
I think you are correct to look at activity though. Will activity will spike back up after manipulation? Perhaps someone should try noise trading a couple inactive markets as an experiment.
Polymarket’s brier score, from their accuracy page , along with Manifold’s data broken out by liquidity as found in the comment they linked from their accuracy page