Of course if it’s successful, I will be glad to share. I would love to share the details now, but there are a few good reasons not to do so:
1) Writing only about successful projects improves the signal:noise ratio. However enthusiastic I feel about my game, if it fails, it is an evidence that it was not worth publishing.
2) Writing about it would be a reward to me, and it’s probably bad idea to take one’s reward before one does the work.
1) Writing only about successful projects improves the signal:noise ratio. However enthusiastic I feel about my game, if it fails, it is an evidence that it was not worth publishing.
1) Writing only about successful projects improves the signal:noise ratio. However enthusiastic I feel about my game, if it fails, it is an evidence that it was not worth publishing.
Noticing how something failed is useful. You had some reason to think it would work and it didn’t. If it was just a lark, sure, move on, but if you expected it to work, pause, notice you are confused and try to work out what happened. Maybe it’s a calibration exercise, or maybe there’s a patch.
Also, not everyone returns to give us updates. If you say it didn’t work, we can all update more cleanly than if you go dark and we have to weight the update by how likely not letting us know what happened is caused by a negative result or just other commitments.
Therefore I promise to write about positive or negative results of this experiment in Group Rationality Diary in December 2012 (or sooner, in case of failure).
Of course if it’s successful, I will be glad to share. I would love to share the details now, but there are a few good reasons not to do so:
1) Writing only about successful projects improves the signal:noise ratio. However enthusiastic I feel about my game, if it fails, it is an evidence that it was not worth publishing.
2) Writing about it would be a reward to me, and it’s probably bad idea to take one’s reward before one does the work.
What a lovely justification of publication bias.
Noticing how something failed is useful. You had some reason to think it would work and it didn’t. If it was just a lark, sure, move on, but if you expected it to work, pause, notice you are confused and try to work out what happened. Maybe it’s a calibration exercise, or maybe there’s a patch.
Also, not everyone returns to give us updates. If you say it didn’t work, we can all update more cleanly than if you go dark and we have to weight the update by how likely not letting us know what happened is caused by a negative result or just other commitments.
You are right; and so is gwern.
Therefore I promise to write about positive or negative results of this experiment in Group Rationality Diary in December 2012 (or sooner, in case of failure).