The point is to make your explanations have the possibility to increase your knowledge, rather than just satisfy your explanation-itch. If they can equally explain all outcomes, they aren’t really explanations.
To use Eliezer’s favorite example, phlogiston “feels” like an explanation for why things burn—but it doesn’t actually effect what you expect to see happen in the world.
An explanation cannot increase your knowledge.Your knowledge can only increase by observation. Increasing your knowledge is a decision theory problem (exploration/exploitation for example).
Phlogiston explains why some categories of things burn and some don’t. Phlogiston predicts that dry wood will always burn when heated to a certain temperature. Phlogiston explains why different kind of things burn as opposed to sometime burn and sometimes not burn. It explains that if you separate a piece of woods in smaller pieces, every smaller piece will also burn.
To clarify my original point, the problem isn’t the narrative. The narrative is a heuristic, it’s a method to update from an observation by remembering a simple unimodal distribution centered on the narrative (what I think most likely happened, how confident I am)
Edited my reply to correct and clarify (though I’ll pass on debating the merits of phlogiston theory).
After re-reading your original comment (it took me a while to parse it) I generally agree with your points. In particular I think “The bug is discarding the rest of the probability distribution” is a good way of summarizing the problem, and something I’ll be mulling over.
That portion could probably stand to be clarified—at the very least I should provide a link to what I’m referring to: http://yudkowsky.net/rational/technical
The point is to make your explanations have the possibility to increase your knowledge, rather than just satisfy your explanation-itch. If they can equally explain all outcomes, they aren’t really explanations.
To use Eliezer’s favorite example, phlogiston “feels” like an explanation for why things burn—but it doesn’t actually effect what you expect to see happen in the world.
An explanation cannot increase your knowledge.Your knowledge can only increase by observation. Increasing your knowledge is a decision theory problem (exploration/exploitation for example).
Phlogiston explains why some categories of things burn and some don’t. Phlogiston predicts that dry wood will always burn when heated to a certain temperature. Phlogiston explains why different kind of things burn as opposed to sometime burn and sometimes not burn. It explains that if you separate a piece of woods in smaller pieces, every smaller piece will also burn.
To clarify my original point, the problem isn’t the narrative. The narrative is a heuristic, it’s a method to update from an observation by remembering a simple unimodal distribution centered on the narrative (what I think most likely happened, how confident I am)
Edited my reply to correct and clarify (though I’ll pass on debating the merits of phlogiston theory).
After re-reading your original comment (it took me a while to parse it) I generally agree with your points. In particular I think “The bug is discarding the rest of the probability distribution” is a good way of summarizing the problem, and something I’ll be mulling over.