I like this idea, and am particularly happy about you building a small prototype to test it. I mostly share Ray’s scepticism, and also expect that conditional probabilities that condition on more than three independent events will be extremely hard to vote on, and will generally be quite badly calibrated. I remember Eliezer having a post on the difficulty of explicitly assessing probabilities for long chains of conditionals (it’s on FB, so who knows whether that link is going to work): https://www.facebook.com/yudkowsky/posts/10154036150109228
I understand the concern, but I’m hoping that as claims about the future are explicit and distinguished from conditional claims, this might not be a problem. That is, if the user has already set P(Trump nominated) = 0.01 and P(Trump president) = 0.009, they will be satisfied with having rejected the claims, and will be able to consider that P(Trump is president | Trump was nominated) = 0.8, in isolation. Also, the conditional P(Trump was nominated | Trump is president) is obviously almost 1, and that should prevent anyone from setting P(Trump is president | Trump was nominated) too low. Also, P(Trump nominated) and P(Trump president) should always have reasonable values on prediction markets, which would set some reasonable bounds on the conditionals.
More generally, I suspect that the Multiple-Stage Fallacy comes from confusing event probabilities with conditional probabilities, and ultimately, I believe that all problems and confusions can be solved with more rigour.
I like this idea, and am particularly happy about you building a small prototype to test it. I mostly share Ray’s scepticism, and also expect that conditional probabilities that condition on more than three independent events will be extremely hard to vote on, and will generally be quite badly calibrated. I remember Eliezer having a post on the difficulty of explicitly assessing probabilities for long chains of conditionals (it’s on FB, so who knows whether that link is going to work): https://www.facebook.com/yudkowsky/posts/10154036150109228
I understand the concern, but I’m hoping that as claims about the future are explicit and distinguished from conditional claims, this might not be a problem. That is, if the user has already set P(Trump nominated) = 0.01 and P(Trump president) = 0.009, they will be satisfied with having rejected the claims, and will be able to consider that P(Trump is president | Trump was nominated) = 0.8, in isolation. Also, the conditional P(Trump was nominated | Trump is president) is obviously almost 1, and that should prevent anyone from setting P(Trump is president | Trump was nominated) too low. Also, P(Trump nominated) and P(Trump president) should always have reasonable values on prediction markets, which would set some reasonable bounds on the conditionals.
More generally, I suspect that the Multiple-Stage Fallacy comes from confusing event probabilities with conditional probabilities, and ultimately, I believe that all problems and confusions can be solved with more rigour.