Launching Forecast, a community for crowdsourced predictions from Facebook

Rebecca Kossnick20 Oct 2020 6:20 UTC
111 points
14 comments3 min readLW link

Greetings! By way of brief introduction, I’m a long time lurker on LessWrong and a member of the team building Forecast, a crowdsourced prediction and discussion community. Full disclosure: we’re part of a labs group, NPE, inside of Facebook.

We launched publicly a few weeks ago. I wanted to share a little about what we’re trying to build and get this community’s advice on where we’re going.

First, a bit of background

Forecast is built around a prediction market: an exchange where people use virtual points to trade on outcomes of future events. Forecast is unique in a few ways:

  1. Members of the community ask all the questions, which range from serious to arcane and mostly center around current events. Our team currently moderates the questions (mostly to edit them for clarity). Over time, we want to empower the community to self-moderate.

  2. Forecast uses points, rather than real money. Forecasters get points when they join and then regularly get refreshes as they play. There’s a leaderboard that tracks total point ‘profit’.

  3. All activity is public: anyone, whether or not they’re a forecast user, can see who participated and what their transaction/​discussion history is. Forecast accounts are tied to your Facebook account behind the scenes, but users can select non-identifying display names to use in the community.

  4. Post-forecast discussion is incentivized. Users get points if other users support the reasons they write to explain their forecasts. Only people who have made a forecast on a particular question can write and support reasons.

Since June, the Forecast community has made more than 50,000 forecasts on a few hundred questions—and they’re actually reasonably accurate. Forecast’s midpoint brier score (measured at the midpoint between a question’s launch and resolution dates) across all closed Forecasts over the past few months is 0.204, compared to Good Judgement’s published result of 0.227 for prediction markets. [Update: as one commenter pointed out, this does not control for time horizon or difficulty, so they’re not really comparable metrics. Leaving the comparison, but taking edited to make this clear.]

Why we’re building this

Like others in this community, we became interested in prediction markets because, when they work, they help push participants to be rational, even at the expense of being popular.

Beyond the forecasts themselves, we think what makes Forecast interesting is the discussion people are having there. While sharing reasoning behind your forecast isn’t a required part of participating, it’s both highly encouraged and explicitly incentivized (you get extra forecasting points if others support your reasons). So far, the discussion in Forecast has been both thoughtful and measured. We believe that the incentive mechanic plus the lack of real money on the line plus the initially small, supportive beta community have driven this trend.

This is what got us excited about building Forecast in the first place: The idea that a forecasting frame and a few well-placed incentives could help foster more rational conversation. We believe that this frame can be particularly impactful in anchoring—and ultimately, hopefully rationalizing—discussion around contentious current events. In a period of increasing uncertainty and division, what we want most is to create a space in the world where people, especially those with divergent views, can find common ground.

Where is this going?

Our first priority is to build something that’s really fun for people who want to engage in rational debate about the future. We’ve built a small community so far and we’re focused on continuing to make it more rewarding to debate and contribute to the collective understanding of what might happen.

Beyond the participating community, we think the forecasts, debate, and analysis in the app could also be valuable to a broader audience. This could be useful both as a novel aggregator of interesting news, opinions and predictions, and as a dataset to help people understand how collective understanding of an event has changed over time. We’re actually planning to release the full dataset (forecasts and discussion, all anonymized) next week. Shoot me an email if you’d like a heads up when this goes live.

At scale, we think that the points system in Forecast could serve as a new form of expert credentialing, both for forecasters and for the information itself. If, for example, someone makes consistently great forecasts on economic news, maybe that could give them access to extra responsibilities or distribution on economic topics. Similarly, the frequency with which a source is cited by the best forecasters could be used as one signal to determine the quality of its content.

Tell us what you think!

Forecast is a work in progress. We’d love to hear what you think of the idea. I’ll be around in the comments and you can also reach out privately to our team if you prefer.

You’re also welcome to join the community. The main discussions take place in our iOS app, and we also have a more viewer-friendly (and less feature-rich) website. We know there’s a lot that needs improving here. We’re just trying to learn and make every day better in Forecast than the one before!

Final note: if you’d like to follow along and get longer form updates on what we’re up to, please feel free to subscribe to our newsletter.

Rebecca Kossnick20 Oct 2020 6:20 UTC
111 points
14 comments3 min readLW link