I’m by no means an expert in my old field (distributed computing) or my new one (technical AI Safety), but you might push me to try forecasting more by default, so that if I’m ever an expert that people listen to, I’ll be used to doing what is my responsibility.
That makes me think: maybe a culture where everyone makes verifiable forecasts would be one that judges expert on how good their forecasts are? Not that I know how to push such a culture.
That makes me think: maybe a culture where everyone makes verifiable forecasts would be one that judges expert on how good their forecasts are? Not that I know how to push such a culture.
I’ve been thinking a lot about this. One thing I came up with was Reddit + Prediction Market. Instead of winning/losing money based on predictions, you get win/lose ability to influence content on the site. Front page content would be based on what correct predictors want to see, rather than all votes being equal.
We do have something very loosely like this here. Unlike on Reddit, getting more karma on LW gives your own posts more starting karma, and also ups the amount of Karma you’re able to add/remove from posts. Being able to write well-received posts gives you more of an ability to highlight or bury other people’s posts.
It would be neat if there was a forecasting equivalent to Facebook. One unified identity associated with various prediction markets. Perhaps with lots of tools to create specialized forecasts. You could imagine that there was an official “City of New York” forecasting page, in which the government could post questions that citizens could make forecast about the effect of bills, conditional on them passing or not passing.
Other websites could tie into… let’s call it “Castbook”… and gauge the amount of influence that accounts received on their site based on their track record on Castbook. Castbook could be a professional-esque website like LinkedIn, so that employers would consider who to hire based on their track record.
At its highest growth potential, I imagine that this would be sort of like a “light side” version of China’s reputation tracking system. It would be voluntary, and be about making forecasts rather than judging people. But it would allow this tremendous resource of people considering outcomes to be distilled, preserved, and used not only to make decisions, but also as a complement or alternative to our present credentialing system.
I actually didn’t realize that is how karma works here. Thanks. I think the critique would be that, as I understand it, we would be rewarding popularity rather than accuracy with this model.
Well, the key challenge with forecasting is that you have to choose questions that can be empirically verified.
Most of the posts here aren’t forecasts, though some are, and those often do have explicit forecasts.
When they don’t contain forecasts, sometimes they imply them.
I think a “strong” post implies many plausible forecasts, or gives a reasonable suggestion as to how to make them.
For example, Eliezer has a post titled “belief in belief” that advises that beliefs should pay rent in terms of concrete predictions. This doesn’t obviously imply an easy-to-empirically-evaluate prediction, except that readers may be sympathetic to this idea and find it useful.
I’m confused about the difference of your proposal with karma. Is it basically that predictions give/remove karma when they’re respectively correct/incorrect, and thus that a high karma post would probably represent what most best predictors want to see?
Most internet forums are pretty pure democracies. If you can get an account, you can upvote/downvote as much as anyone else and each post you make is graded in a vacuum. I’m proposing an alternative that does not treat all participants equally.
Accurate predictions give you more karma and that karma means your posts/comments/upvotes/downvotes rank higher and have more influence than people who have not built up “accuracy karma”
Thanks for this post!
I’m by no means an expert in my old field (distributed computing) or my new one (technical AI Safety), but you might push me to try forecasting more by default, so that if I’m ever an expert that people listen to, I’ll be used to doing what is my responsibility.
That makes me think: maybe a culture where everyone makes verifiable forecasts would be one that judges expert on how good their forecasts are? Not that I know how to push such a culture.
I’ve been thinking a lot about this. One thing I came up with was Reddit + Prediction Market. Instead of winning/losing money based on predictions, you get win/lose ability to influence content on the site. Front page content would be based on what correct predictors want to see, rather than all votes being equal.
We do have something very loosely like this here. Unlike on Reddit, getting more karma on LW gives your own posts more starting karma, and also ups the amount of Karma you’re able to add/remove from posts. Being able to write well-received posts gives you more of an ability to highlight or bury other people’s posts.
It would be neat if there was a forecasting equivalent to Facebook. One unified identity associated with various prediction markets. Perhaps with lots of tools to create specialized forecasts. You could imagine that there was an official “City of New York” forecasting page, in which the government could post questions that citizens could make forecast about the effect of bills, conditional on them passing or not passing.
Other websites could tie into… let’s call it “Castbook”… and gauge the amount of influence that accounts received on their site based on their track record on Castbook. Castbook could be a professional-esque website like LinkedIn, so that employers would consider who to hire based on their track record.
At its highest growth potential, I imagine that this would be sort of like a “light side” version of China’s reputation tracking system. It would be voluntary, and be about making forecasts rather than judging people. But it would allow this tremendous resource of people considering outcomes to be distilled, preserved, and used not only to make decisions, but also as a complement or alternative to our present credentialing system.
I actually didn’t realize that is how karma works here. Thanks. I think the critique would be that, as I understand it, we would be rewarding popularity rather than accuracy with this model.
Yeah, and that’s not necessarily a bad thing. In theory, it rewards people who are strong content producers, rather than critics.
Undoubtedly true.
However. I suspect most here would claim they want accurate content rather than “strong” content.
We should be candid about what we are optimizing for either way.
Well, the key challenge with forecasting is that you have to choose questions that can be empirically verified.
Most of the posts here aren’t forecasts, though some are, and those often do have explicit forecasts.
When they don’t contain forecasts, sometimes they imply them.
I think a “strong” post implies many plausible forecasts, or gives a reasonable suggestion as to how to make them.
For example, Eliezer has a post titled “belief in belief” that advises that beliefs should pay rent in terms of concrete predictions. This doesn’t obviously imply an easy-to-empirically-evaluate prediction, except that readers may be sympathetic to this idea and find it useful.
Do you mean for pushing this culture on LW?
I’m confused about the difference of your proposal with karma. Is it basically that predictions give/remove karma when they’re respectively correct/incorrect, and thus that a high karma post would probably represent what most best predictors want to see?
I think you’re interpreting me correctly.
Most internet forums are pretty pure democracies. If you can get an account, you can upvote/downvote as much as anyone else and each post you make is graded in a vacuum. I’m proposing an alternative that does not treat all participants equally.
Accurate predictions give you more karma and that karma means your posts/comments/upvotes/downvotes rank higher and have more influence than people who have not built up “accuracy karma”