Certainly, for me, there has always been a gap between how useful I think predictions are, in theory, and how often I make them.
Same here.
There’s probably some additional value you can extract by writing down quantitative confidence levels, and by devising objective metrics that are impossible to game, rather than just relying on your subjective impressions.
I agree.
But in most cases I don’t think that additional value is worth the cost you incur from turning predictions into an onerous task.
I disagree in that (1) I think much of the value of predictions would come from the ability to examine and analyze my past prediction accuracy and (2) I don’t think the task of recording the predictions would necessarily be very onerous (e.g. especially if there is some recurring prediction which you don’t have to write a new description for every time you make it).
I really like Prediction Book (which I just checked out for the first time before Googling and finding this post), but it doesn’t offer sufficient analysis options to make me want to really begin using it yet.
But this could change!
I would predict (75%) that I would begin using it on a daily basis (and would continue to do so indefinitely upon realizing that I was indeed getting sufficient value out of it to justify the time it takes to record my predictions on the site) if it offered not just the single Accuracy vs 50-100% Confidence plot and graph, but the following features:
Ability to see confidence and accuracy plotted versus time. (Useful, e.g. to see weekly progress on meeting some daily self-imposed deadline. Perhaps you used to meet it 60% of days on average, but now you meet it 80% of days on average. You could track your progress while seeing if you accurately predict progress as well, or if your predicted values follow the improvement.)
Ability to see 0-100% confidence on statistics plot, instead of just 50-100%. (Maybe it already includes 0-50% and just does the negative of each prediction (?). However, if so, this is still a problem since I may have different biases for 10% predictions than 90% predictions.)
Ability to set different different prediction types and analyze the data separately. (Useful for learning how accurate one’s predictions are in different domains.)
Ability to download all of one’s past prediction data. (Useful if there is some special analysis that one wants to perform.)
A public/private prediction toggle button (Useful because there may be times when it’s okay for someone to hide a prediction they were embarrassingly wrong about or someone may want to publicize a previously-private prediction. Forcing users to decide at the time of the prediction whether their prediction will forever be displayed publicly on their account or remain private forever doesn’t seem very user-friendly.)
Bonus: An app allowing easy data input when not at your computer. (Would make it even more convenient to record predictions.)
Some of these features can be achieved by creating multiple accounts. And I could accomplish all of this in Excel. But using multiple accounts or Excel would make it too tedious to be worth it. The value is in having the graphs and analysis automatically generated and presented to you with only a small amount of effort needed to input the predictions in the first place.
I don’t think any of these additional features would be very difficult to implement. However, I’m not a programmer, so for me to dive into the Prediction Book GitHub and try to figure out how to make these changes would probably be quite time-consuming and not worth it.
Maybe there is someone else who agrees that these features would be useful to them who is a programmer and would like to add some or all of the suggested features I mentioned? Does anyone know the people who did most of the work programming the current website?
Same here.
I agree.
I disagree in that (1) I think much of the value of predictions would come from the ability to examine and analyze my past prediction accuracy and (2) I don’t think the task of recording the predictions would necessarily be very onerous (e.g. especially if there is some recurring prediction which you don’t have to write a new description for every time you make it).
I really like Prediction Book (which I just checked out for the first time before Googling and finding this post), but it doesn’t offer sufficient analysis options to make me want to really begin using it yet.
But this could change!
I would predict (75%) that I would begin using it on a daily basis (and would continue to do so indefinitely upon realizing that I was indeed getting sufficient value out of it to justify the time it takes to record my predictions on the site) if it offered not just the single Accuracy vs 50-100% Confidence plot and graph, but the following features:
Ability to see confidence and accuracy plotted versus time. (Useful, e.g. to see weekly progress on meeting some daily self-imposed deadline. Perhaps you used to meet it 60% of days on average, but now you meet it 80% of days on average. You could track your progress while seeing if you accurately predict progress as well, or if your predicted values follow the improvement.)
Ability to see 0-100% confidence on statistics plot, instead of just 50-100%. (Maybe it already includes 0-50% and just does the negative of each prediction (?). However, if so, this is still a problem since I may have different biases for 10% predictions than 90% predictions.)
Ability to set different different prediction types and analyze the data separately. (Useful for learning how accurate one’s predictions are in different domains.)
Ability to download all of one’s past prediction data. (Useful if there is some special analysis that one wants to perform.)
A public/private prediction toggle button (Useful because there may be times when it’s okay for someone to hide a prediction they were embarrassingly wrong about or someone may want to publicize a previously-private prediction. Forcing users to decide at the time of the prediction whether their prediction will forever be displayed publicly on their account or remain private forever doesn’t seem very user-friendly.)
Bonus: An app allowing easy data input when not at your computer. (Would make it even more convenient to record predictions.)
Some of these features can be achieved by creating multiple accounts. And I could accomplish all of this in Excel. But using multiple accounts or Excel would make it too tedious to be worth it. The value is in having the graphs and analysis automatically generated and presented to you with only a small amount of effort needed to input the predictions in the first place.
I don’t think any of these additional features would be very difficult to implement. However, I’m not a programmer, so for me to dive into the Prediction Book GitHub and try to figure out how to make these changes would probably be quite time-consuming and not worth it.
Maybe there is someone else who agrees that these features would be useful to them who is a programmer and would like to add some or all of the suggested features I mentioned? Does anyone know the people who did most of the work programming the current website?