SimonM

Karma: 1,004

SimonM 22 Jan 2024 11:51 UTC
2 points
0
on: Why have insurance markets succeeded where prediction markets have not?
I wrote about exactly this recently- https://www.lesswrong.com/posts/zLnHk9udC28D34GBB/prediction-markets-aren-t-magic

SimonM 9 Jan 2024 17:23 UTC
2 points
0
in reply to: Dagon’s comment on: Hiring decisions are not suitable for prediction markets

I don’t give much weight to his diagnosis of problematic group decision mechanisms

I have quite a lot of time for it personally.

The world is dominated by a lot of large organizations that have a lot of dysfunction. Anybody over the age of 40 will just agree with me on this. I think it’s pretty hard to find anybody who would disagree about that who’s been around the world. Our world is full of big organizations that just make a lot of bad decisions because they find it hard to aggregate information from all the different people.

This is roughly Hanson’s reasoning, and you can spell out the details a bit more. (Poor communication between high level decision makers and shop-floor workers, incentives at all levels dissuading truth telling etc). Fundamentally though I find it hard to make a case this isn’t true in /any/ large organization. Maybe the big tech companies can make a case for this, but I doubt it. Office politics and self-interest are powerful forces.

For employment decisions, it’s not clear that there is usable (legally and socially tolerated) information which a market can provide

I roughly agree—this is the point I was trying to make. All the information is already there in interview evaluations. I don’t think Robin is expecting new information though—he’s expecting to combine the information more effectively. I just don’t expect that to make much difference in this case.

Hiring decisions are not suitable for prediction markets

SimonM8 Jan 2024 21:11 UTC

12 points

6 comments1 min readLW link

SimonM 24 Dec 2023 12:14 UTC
1 point
0
in reply to: Alexander Gietelink Oldenziel’s comment on: Prediction Markets aren’t Magic
So the first question is: “how much should we expect the sample mean to move?”.
If the current state is $μ_{n}$ , and we see a sample of $x$ (where $x$ is going to be 0 or 1 based on whether or not we have heads or tails), then the expected change is:
$E ((μ_{n} - μ_{n + 1})^{2}) = \frac{1}{(n + 1)^{2}} E (μ_{n} - x) = \frac{1}{(n + 1)^{2}} (μ_{n}^{2} - 2 μ_{n} E (x) + E (x^{2}))$
$\dots = \frac{1}{(n + 1)^{2}} (μ_{n}^{2} - 2 μ_{n} μ_{n} + μ_{n}^{2} + σ_{x}^{2}) = \frac{σ_{x}^{2}}{(n + 1)^{2}} \sim O (\frac{1}{n^{2}})$
In these steps we are using the facts that ( $x$ is independent of the previous samples, and the distribution of $x$ is Bernoulli with $p = μ_{n}$ . (So $E (x) = μ_{n}$ and $V a r (x) = σ_{x}^{2} = μ_{n} (1 - μ_{n})$ ).
To do the proper version of this, we would be interested in how our prior changes, and our distribution for $x$ wouldn’t purely be a function of $μ_{n}$ . This will reduce the difference, so I have glossed over this detail.
The next question is: “given we shift the market parameter by $O (1 / n^{2})$ , how much money (pnl) should we expect to be able to extract from the market in expectation?”
For this, I am assuming that our market is equivalent to a proper scoring rule. This duality is laid out nicely here. Expending the proper scoring rule out locally, it must be of the form $f (x) = c + O (x^{2})$ , since we have to be at a local minima. To use some classic examples, in a log scoring rule:
$q log (p) + (1 - q) log (1 - p) = (q log (q) - (q - 1) log (1 - q)) + \frac{(p - q)^{2}}{(2 (q - 1) q)} + O ((p - q)^{3})$
in a brier scoring rule:
$q (1 - p)^{2} + (1 - q) p^{2} = q (1 - q)^{2} + (1 - q) q^{2} + (q - p)^{2}$
What links here?
- Prediction Markets aren’t Magic by SimonM (21 Dec 2023 12:54 UTC; 90 points)

SimonM 24 Dec 2023 12:13 UTC
1 point
0
in reply to: Throwaway2367’s comment on: Prediction Markets aren’t Magic
Whoops. Good catch. Fixing

SimonM 23 Dec 2023 17:03 UTC
1 point
0
in reply to: Alexander Gietelink Oldenziel’s comment on: Prediction Markets aren’t Magic
x is the result of the (n+1)th draw sigma is the standard deviation after the first n draws pnl is the profit and loss the bettor can expect to earn

SimonM 22 Dec 2023 9:19 UTC
4 points
0
in reply to: Ebenezer Dukakis’s comment on: Prediction Markets aren’t Magic
Prediction markets generate information. Information is valuable as a public good. Failure of public good provision is not a failure of prediction markets.
I think you’ve slightly missed my point. My claim is narrower than this. I’m saying that prediction markets have a concrete issue which means you should expect them to be less efficient at gathering data than alternatives. Even if information is a public good, it might not be worth as much as prediction markets would charge to find that information. Imagine if the cost of information via a prediction market was exponential in the cost of information gathering, that wouldn’t mean the right answer is to subsidise prediction markets more.

SimonM 22 Dec 2023 9:15 UTC
1 point
0
in reply to: mako yass’s comment on: Prediction Markets aren’t Magic
If you have another suggestion for a title, I’d be happy to use it

SimonM 22 Dec 2023 9:14 UTC
2 points
1
in reply to: jacob_cannell’s comment on: Prediction Markets aren’t Magic
Even if there is no acceptable way to share the data semi-anonymously outside of match group, the arguments for prediction markets still apply within match group. A well designed prediction market would still be a better way to distribute internal resources and rewards amongst competing data science teams within match group.
I used to think things like this, but now I disagree, and actually think it’s fairly unlikely this is the case.
1. Internal prediction markets have tried (and failed) at multiple large organisations who made serious efforts to create them
2. As I’ve explained in this post, prediction markets are very inefficient at sharing rewards. Internal to a company you are unlikely to have the right incentives in place as much as just subsidising a single team who can share models etc. The added frictions of a market are substantial.
3. The big selling points of prediction markets (imo) come from:
  1. Being able to share results without sharing information (ie I can do some research, keep the information secret, but have people benefit from the conclusions)
  2. Incentivising a wider range of people. At an orgasation, you’d hire the most appropriate people into your data science team and let them run. There’s no need to wonder if someone from marketing is going to outperform their algorithm.
People who actually match and meetup with another user will probably have important inside view information inaccessible to the algorithms of match group.
I strongly agree. I think people often confuse “market” and “prediction market”. There is another (arguably better) model of dating apps which is that the market participants are the users, and the site is actually acting as a matching engine. Since I (generally) think markets are great, this also seems pretty great to me.

SimonM 21 Dec 2023 14:12 UTC
3 points
0
in reply to: quetzal_rainbow’s comment on: Prediction Markets aren’t Magic
Sure—but that answer doesn’t explain their relative lack of success in other countries (eg the UK)

Additionally, where prediction markets work well (eg sports betting, political betting) there is a thriving offshore market catering to US customers.

Prediction Markets aren’t Magic

SimonM21 Dec 2023 12:54 UTC

90 points

29 comments3 min readLW link

SimonM 21 Dec 2023 12:53 UTC
3 points
0
in reply to: Saul Munn’s comment on: Solving Two-Sided Adverse Selection with Prediction Market Matchmaking
This post triggered me a bit, so I ended up writing one of my own.
I agree the entire thing is about how to subsidise the markets, but I think you’re overestimating how good markets are as a mechanism for subsidising forecasting (in general). Specifically for your examples:
1. Direct subsidies are expensive relative to the alternatives (the point of my post)
2. Hedging doesn’t apply in lots of markets, and in the ones where it does make sense those markets already exist. (Eg insurance)
3. New traders is a terrible idea as you say. It will work in some niches (eg where there’s lots of organic interest, but it wont work at scale for important things)

SimonM 27 Nov 2023 11:22 UTC
3 points
1
on: Solving Two-Sided Adverse Selection with Prediction Market Matchmaking

I’m excited about the potential of conditional prediction markets to improve on them and solve two-sided adverse selection.

This applies to roughly the entire post, but I see an awful lot of magical thinking in this space. What is the actual mechanism by which you think prediction markets will solve these problems?

In order to get a good prediction from a market you need traders to put prices in the right places. This means you need to subsidise the markets. Whether or not a subsidised prediction market is going to be cheaper for the equivalent level of forecast than paying another 3rd party (as is currently the case in most of your examples) is very unclear to me

SimonM 23 Nov 2023 17:25 UTC
4 points
0
on: AI #39: The Week of OpenAI

A thing Larry Summers once said that seems relevant, from Elizabeth Warren:

He said something very similar to Yanis Varoufakis (https://www.globaljustice.org.uk/blog/2017/06/when-yanis-met-prince-darkness-extract-adults-room/) and now I like to assume he goes around saying this to everyone

SimonM 8 Nov 2023 15:19 UTC
1 point
0
in reply to: PaulK’s comment on: When and why should you use the Kelly criterion?
No, it’s fairly straightforward to see this won’t work

Let N be the random variable denoting the number of rounds. Let x = p*w+(1-p)*l where p is probability of winning and w=1-f+o*f, l=1-f the amounts we win or lose betting a fraction f of our wealth.

Then the value we care about is E[x^N], which is the moment generating function of X evaluated at log(x). Since our mgf is increasing as a function of x, we want to maximise x. ie our linear utility doesn’t change

SimonM 8 Nov 2023 7:13 UTC
1 point
0
in reply to: Jonas V’s comment on: How to (hopefully ethically) make money off of AGI
Yes? 1/ it’s not in their mandate 2/ they’ve never done it before (I guess you could argue the UK did for in 2022, but I’m not sure this is quite the same) 3/ it’s not clear that this form of QE would have the effect you’re expecting on long end yields

SimonM 7 Nov 2023 21:30 UTC
3 points
−1
in reply to: NoSignalNoNoise’s comment on: How to (hopefully ethically) make money off of AGI
I absolutely do not recommend shorting long-dated bonds. However, if I did want to do so a a retail investor, I would maintain a rolling short in CME treasury futures. Longest future is UB. You’d need to roll your short once every 3 months, and you’d also want to adjust the size each time, given that the changing CTD means that the same number of contracts doesn’t necessarily mean the same amount of risk each expiry.

SimonM 7 Nov 2023 21:24 UTC
3 points
1
in reply to: Jonas V’s comment on: How to (hopefully ethically) make money off of AGI
Err… just so I’m clear lots of money being printed will devalue those long dated bonds even more, making the bond short an even better trade? (Or are you talking about some kind of YCC scenario?)

SimonM 6 Nov 2023 9:08 UTC
1 point
0
in reply to: AlexMennen’s comment on: When and why should you use the Kelly criterion?

average returns

I think the disagreement here is on what “average” means. All-in maximises the arithmetic average return. Kelly maximises the geometric average. Which average is more relevant is equivalent to the Kelly debate though, so hard to say much more

SimonM 21 Sep 2023 15:38 UTC
2 points
0
on: AI #30: Dalle-3 and GPT-3.5-Instruct-Turbo
Wouldn’t You Prefer a Good Game of Chess?
I assume this was supposed to be a WarGames reference, in which case I think it should be a “nice” game of chess.

SimonM

Hiring de­ci­sions are not suit­able for pre­dic­tion markets

Pre­dic­tion Mar­kets aren’t Magic

Hiring decisions are not suitable for prediction markets

Prediction Markets aren’t Magic