I found this example very confusing, but perhaps it was confusing in a good way because I had to spend quite a while to grasp exactly what the argument was.
Unless I am still misunderstanding your argument, the second market proposal you make is exactly what Hanson’s solution to this is, and it’s mentioned on the page for futarchy here:
A betting market can estimate whether a proposed policy would increase national welfare by comparing two conditional estimates: national welfare conditional on adopting the proposed policy, and national welfare conditional on not adopting the proposed policy. Betting markets can produce conditional estimates several ways, such as via “called-off bets,” i.e., bets that are called off if a condition is not met.
I think your original conjunctive market is straightforwardly a bad idea, as you were able to determine. So now we are left with your practical problems.
This procedure will get the correct results in theory, but it has the following practical problems: It allows maximizing on only one outcome metric (because one cannot precommit to choose the President based on criteria that could potentially be inconsistent with each other). Moreover, it requires the reversal of trades, which will be problematic if people who won money on the Jeb contract have withdrawn their winnings from the exchange.
It allows only maximizing on one outcome metric, but that outcome metric can be in theory as complex as you want. If you could encode your entire utility function into the outcome metric then that’s not a big deal. More concretely, for likely uses of futarchy any outcome measure would be better than the status quo, because right now, for example, project funding is determined by the whims of politicians to curry favor with an uneducated electorate. How many stadiums would be built in cities if they had to pass the crucible of actually having economic actors believe in the supposed economic gains the stadium would bring to the city? (Being nuked is likely to effect many summary measures that could plausibly be used to elect presidents, like GDP + some measure of social well-being)
Reversal of trades is not problematic, at least for the way you are thinking it is. Sports betting establishments call off trades all the time, such as when a game is cancelled due to inclement weather. People who are able to withdraw money have already made their contribution to the market, and when they sold their contracts someone else bought it and still has it open. The only reason this would seem bad is because of some sense of cosmic justice, where one doesn’t like people who are wrong to make money or something. Their are more realistic concerns circling around this one regarding liquidity, opportunity costs for holding trades, and margin accounts, but they are largely open issues until we have more prediction markets.
A betting market can estimate whether a proposed policy would increase national welfare by comparing two conditional estimates: national welfare conditional on adopting the proposed policy, and national welfare conditional on not adopting the proposed policy. Betting markets can produce conditional estimates several ways, such as via “called-off bets,” i.e., bets that are called off if a condition is not met.
This looks like a restatement of the original market I posted? The choice of president is the “proposed policy” and the nuclear attack is the measure of “national welfare”. The fact that he uses causal language (“increase”) in this setting is unfortunate. My first proposal did not have called-off bets, but this is not itself sufficient to solve the problem.
Reversal of trades is not problematic, at least for the way you are thinking it is. Sports betting establishments call off trades all the time, such as when a game is cancelled due to inclement weather. People who are able to withdraw money have already made their contribution to the market, and when they sold their contracts someone else bought it and still has it open. The only reason this would seem bad is because of some sense of cosmic justice
I will have to think more about whether trade reversal, it is possible that you are right and that it is not necessary.
For the moment, I will just note that I did not suggest trade reversal for reasons of fairness; the reason is that this reversal procedure is necessary in order to incentivize participants to trade on their beliefs about causal quantities. If traders believe that the prediction market on Jeb will be reversed if Hillary is chosen, then the prices they are willing to pay will only reflect their beliefs about what happens if Jeb is chosen (assuming that the only cause of who is elected is the predicton market, which is why I insisted on precommitment). The alternative to reversing the trades is to expire the contract at the last traded value, but if traders expect this, the price they are willing to pay may become entangled with their beliefs about who will be chosen (which may be affected by confounders)
That was the logic behind it, but please give me a couple of days to think about whether it is correct
His conditional bets include called-off bets, what you call reversal of trades, which is why I thought what he is talking about corresponds to your second market proposal. Your first market proposal doesn’t have called-off bets. Is there some part of your statistical objection which isn’t solved by called-off bets?
On further thought, I think the reversal of trades going back to all the traders in the market may be necessary, while I was only thinking that a reversal of outstanding contracts would be necessary. That could lead to losses to the exchange if, for example, the only remaining holders had acquired their contracts at expensive times, so the $100 that exists to settle both contracts is insufficient to reverse both trades. As alternatives, the exchange could (1) disallow withdrawals until the contract is fully settled, (2) subsidize those losses through trading fees, or (3) only pay out reversals pro rata to the extent funds are available. The first wouldn’t distort the market but may affect liquidity. In general I’m skeptical of proposals that have extremely long time frames, but Hanson seems to want contracts that are decades long in some places, as I recall. The second would distort the market a little bit because of the fees, but it’s likely to be by a manageable amount. The third would probably distort the market the most, especially as it got to the decision time, and liquidity might dry up if there was a perception that the current holders had all gotten in at high prices. I think in general Hanson sees prediction markets as being subsidized markets, which 2 is most in-line with, perhaps without even the trading fees depending on the magnitude of the subsidy.
His conditional bets include called-off bets, what you call reversal of trades, which is why I thought what he is talking about corresponds to your second market proposal. Your first market proposal doesn’t have called-off bets. Is there some part of your statistical objection which isn’t solved by called-off bets?
I see where you are coming from, in fact this used to be my position: This post was inspired by a talk I gave at the Less Wrong Meetup, where I made the claim that the causality problem is solved by called-off bets (reversals). Jimrandomh called me on it, and he was right: There will still be confounding even when bets are called off/reversed:
Imagine there are two possible worlds: In one of them, Kim is overthrown, in the other he is not. I expect the probability of Hillary being elected will be much higher if he is overthrown. I also expect that the probability of an attack is much higher if Kim is still in office.
If I make a bet on the probability of an attack given that Hillary is elected, and I know that this market will only be settled in the case that she is elected, my estimated probability will incorporate information about the fact that if the market is settled, more likely than not, Kim was overthrown. However, at the time I make the bet, I don’t know whether Kim will be overthrown, so this means my bet incorporates information that causally does not depend on Hillary being elected.
We tried to solve this using the “precommitment” mechanism. In graphical terms, the idea is that this removes all arrows into the election by ensuring that the only cause of who gets to be President is the prediction market itself. My intuition is that it works, but it is certainly something that should be doublechecked by someone with more technical expertise on prediction markets and causality.
Deciding the outcome based on the price in the betting market is the whole point of futarchy. You seem to be saying that prediction markets in absence of futarchy don’t provide good advice on how you should vote. That is an interesting point which I hadn’t considered before your post.
I am still uncomfortable with your example, however. If Kim is overthrown prior to the election, the market rates will adjust based on that information. If he’s overthrown after the election, then there is no causal link between that and the election results, presumably. Prior to his overthrow, the market simply provides the best estimate of the outcome until that result is known. All that means is you shouldn’t make decisions based on outdated estimates that didn’t include all known information.
Yes, the point of futarchy is to make the decision based on the price in the prediction market. What I am saying is that if you want participants to provide their best guesses about which decisions will maximize the outcome, you have to make a credible pre-commitment that the only factor that influences the decision is the prediction market that is currently being traded. You can only make such a commitment for one prediction market per decision (but like you say, the outcome measure can be arbitrarily complex)
I think you are right that once it becomes known whether Kim is overthrown, it is no longer a confounder. Therefore, the bias should be expected to get lower the nearer we get to the decision time point. However, some confounders may be unobservable, or unobserved until the time the decision is made. For instance, if this is not a pure futarchy and there is voting going on, you may gain information from the make-up of the electorate. Imagine there is a referendum on a 70% income tax and Bernie Sanders is running for President. Even if he has no influence on whether the referendum passes, his chances of being elected will be correlated with the outcome of the referendum, and you won’t know which state you are in until you see the exit polls.
I found this example very confusing, but perhaps it was confusing in a good way because I had to spend quite a while to grasp exactly what the argument was.
Unless I am still misunderstanding your argument, the second market proposal you make is exactly what Hanson’s solution to this is, and it’s mentioned on the page for futarchy here:
I think your original conjunctive market is straightforwardly a bad idea, as you were able to determine. So now we are left with your practical problems.
It allows only maximizing on one outcome metric, but that outcome metric can be in theory as complex as you want. If you could encode your entire utility function into the outcome metric then that’s not a big deal. More concretely, for likely uses of futarchy any outcome measure would be better than the status quo, because right now, for example, project funding is determined by the whims of politicians to curry favor with an uneducated electorate. How many stadiums would be built in cities if they had to pass the crucible of actually having economic actors believe in the supposed economic gains the stadium would bring to the city? (Being nuked is likely to effect many summary measures that could plausibly be used to elect presidents, like GDP + some measure of social well-being)
Reversal of trades is not problematic, at least for the way you are thinking it is. Sports betting establishments call off trades all the time, such as when a game is cancelled due to inclement weather. People who are able to withdraw money have already made their contribution to the market, and when they sold their contracts someone else bought it and still has it open. The only reason this would seem bad is because of some sense of cosmic justice, where one doesn’t like people who are wrong to make money or something. Their are more realistic concerns circling around this one regarding liquidity, opportunity costs for holding trades, and margin accounts, but they are largely open issues until we have more prediction markets.
This looks like a restatement of the original market I posted? The choice of president is the “proposed policy” and the nuclear attack is the measure of “national welfare”. The fact that he uses causal language (“increase”) in this setting is unfortunate. My first proposal did not have called-off bets, but this is not itself sufficient to solve the problem.
I will have to think more about whether trade reversal, it is possible that you are right and that it is not necessary.
For the moment, I will just note that I did not suggest trade reversal for reasons of fairness; the reason is that this reversal procedure is necessary in order to incentivize participants to trade on their beliefs about causal quantities. If traders believe that the prediction market on Jeb will be reversed if Hillary is chosen, then the prices they are willing to pay will only reflect their beliefs about what happens if Jeb is chosen (assuming that the only cause of who is elected is the predicton market, which is why I insisted on precommitment). The alternative to reversing the trades is to expire the contract at the last traded value, but if traders expect this, the price they are willing to pay may become entangled with their beliefs about who will be chosen (which may be affected by confounders)
That was the logic behind it, but please give me a couple of days to think about whether it is correct
His conditional bets include called-off bets, what you call reversal of trades, which is why I thought what he is talking about corresponds to your second market proposal. Your first market proposal doesn’t have called-off bets. Is there some part of your statistical objection which isn’t solved by called-off bets?
On further thought, I think the reversal of trades going back to all the traders in the market may be necessary, while I was only thinking that a reversal of outstanding contracts would be necessary. That could lead to losses to the exchange if, for example, the only remaining holders had acquired their contracts at expensive times, so the $100 that exists to settle both contracts is insufficient to reverse both trades. As alternatives, the exchange could (1) disallow withdrawals until the contract is fully settled, (2) subsidize those losses through trading fees, or (3) only pay out reversals pro rata to the extent funds are available. The first wouldn’t distort the market but may affect liquidity. In general I’m skeptical of proposals that have extremely long time frames, but Hanson seems to want contracts that are decades long in some places, as I recall. The second would distort the market a little bit because of the fees, but it’s likely to be by a manageable amount. The third would probably distort the market the most, especially as it got to the decision time, and liquidity might dry up if there was a perception that the current holders had all gotten in at high prices. I think in general Hanson sees prediction markets as being subsidized markets, which 2 is most in-line with, perhaps without even the trading fees depending on the magnitude of the subsidy.
I see where you are coming from, in fact this used to be my position: This post was inspired by a talk I gave at the Less Wrong Meetup, where I made the claim that the causality problem is solved by called-off bets (reversals). Jimrandomh called me on it, and he was right: There will still be confounding even when bets are called off/reversed:
Imagine there are two possible worlds: In one of them, Kim is overthrown, in the other he is not. I expect the probability of Hillary being elected will be much higher if he is overthrown. I also expect that the probability of an attack is much higher if Kim is still in office.
If I make a bet on the probability of an attack given that Hillary is elected, and I know that this market will only be settled in the case that she is elected, my estimated probability will incorporate information about the fact that if the market is settled, more likely than not, Kim was overthrown. However, at the time I make the bet, I don’t know whether Kim will be overthrown, so this means my bet incorporates information that causally does not depend on Hillary being elected.
We tried to solve this using the “precommitment” mechanism. In graphical terms, the idea is that this removes all arrows into the election by ensuring that the only cause of who gets to be President is the prediction market itself. My intuition is that it works, but it is certainly something that should be doublechecked by someone with more technical expertise on prediction markets and causality.
Deciding the outcome based on the price in the betting market is the whole point of futarchy. You seem to be saying that prediction markets in absence of futarchy don’t provide good advice on how you should vote. That is an interesting point which I hadn’t considered before your post.
I am still uncomfortable with your example, however. If Kim is overthrown prior to the election, the market rates will adjust based on that information. If he’s overthrown after the election, then there is no causal link between that and the election results, presumably. Prior to his overthrow, the market simply provides the best estimate of the outcome until that result is known. All that means is you shouldn’t make decisions based on outdated estimates that didn’t include all known information.
Yes, the point of futarchy is to make the decision based on the price in the prediction market. What I am saying is that if you want participants to provide their best guesses about which decisions will maximize the outcome, you have to make a credible pre-commitment that the only factor that influences the decision is the prediction market that is currently being traded. You can only make such a commitment for one prediction market per decision (but like you say, the outcome measure can be arbitrarily complex)
I think you are right that once it becomes known whether Kim is overthrown, it is no longer a confounder. Therefore, the bias should be expected to get lower the nearer we get to the decision time point. However, some confounders may be unobservable, or unobserved until the time the decision is made. For instance, if this is not a pure futarchy and there is voting going on, you may gain information from the make-up of the electorate. Imagine there is a referendum on a 70% income tax and Bernie Sanders is running for President. Even if he has no influence on whether the referendum passes, his chances of being elected will be correlated with the outcome of the referendum, and you won’t know which state you are in until you see the exit polls.