David Deutsch: A new way to explain explanation
I’m sure this talk will be of interest, even if most of the ideas that he talks about will be familiar to readers here.
[edit]
In this talk David Deutsch discusses “the most important discovery in human history”; how humanity moved beyond a few hundred thousand years of complete ignorance about the universe. Deutsch attempts to be specific about what led to this change—he concludes that it is the insistence that an explanation be ‘hard to vary’.
Whilst a ‘hard to vary’ explanation is functionally the same as a, more commonly known, Occam’s Razor explanation (since fewer parameters necessarily make a fit harder to vary) the slightly different emphasis might be a useful pedagogical tool. A ‘hard to vary’ explanation will perhaps lead more naturally to questions about strong predictions and falsifiability than Occam’s razor. It also seems harder to misunderstand. As we know, Occam’s razor suffers because of the difference between actual complexity and linguistic complexity, so an explanation like “it’s magic” can appear to be simple. Magic might appear simple, but it will never appear ‘hard to vary’, so students of rationality would have one less pitfall awaiting them.
Deutsch also touches on what constitutes understanding and knowledge and cautions us not to trust predictions that are purely of an extrapolated empirical nature as there is no true understanding contained there.
[/edit]
If you haven’t already read Deutsch’s book “The Fabric of Reality” I’d highly recommend that as well.
Downvoted due to lack of useful summary.
Agreed. Don’t link video/audio unless you post a summary of it in your own words.
That’s a fair point, but I’ve never actually seen it mentioned explicitly. Maybe there should be a ‘tips on writing posts’ post.
Actually, you shouldn’t make any top-level post whose sole contribution is a link unless you summarize it (in the case of an argument) or explain its significance with examples (in the case of an information source like an FAQ).
It’s just that this applies especially for audio and video, which have additional time-commitment and searchability constraints. I’ll make a note in anything I can edit about this guideline.
I really think it’s common sense though: do your homework, and respect others’ patience.
A number of good ideas touched on in this talk. It primarily focuses on how the good concepts are those that are hard to vary, which is more instructive than the usual “prefer simplicity” (but not in the amount of stuff suggested by the theory, not in the diversity of its real-world implications, et cetera, et cetera). It reminds of how the roots of any knowledge necessarily lie in human mind. It makes a welcome emphasis on how the scientific revolution, a mere methodological change, ended a stasis tens of thousands of years long, and hence the importance of understanding this change.
Okay, I saw in the comments (both here and on the TED site) that Deutsch’s point was that good explanations are “hard to vary”, but I didn’t understand what that means.
So I finally saw the talk (after skipping most of it to get to the explanation of explanation), and it turns out Deutsch just means “lacking unnecessary details” when he says “hard to vary”. Which is just the standard point about the conjunction fallacy and how each detail makes your explanation less plausible.
Nothing new here, sorry :-/
He does not mean “lacking unnecessary details”. For example the statements “Everyone just acts in his own interest” or “Everyone is really an altruist” are simple and lack unnecessary details, explain quite a lot, and are consistent with Occam’s razor. But by Deutsch’s criteria they are bad explanation because they are too easy to vary. For example, someone who believes in the self-interest theory could say, “John gave to charity because he would have felt guilty otherwise. So he really was selfish” .
We see that it is easy to change the theory that everyone is selfish to accomodate the case of someone who seems altruistic.
Or someone who believes in the altruist theory could say about John murdering Harry, “Well then, Harry must have been very unhappy.”
The altruist theory and the selfishness theory are simple and explanatory in their own way, but too easy to vary. Similarly the idea that that sexism, feminism, capitalism, communism, parental coercion, environmental disregard, etc. cause unhappiness or mental illness or some other broad conclusion are equally meaningless. These explanations are bad because they can be varied to explain ANYTHING.
In contrast, theories that are difficult to vary go out on a limb. They are bold conjectures that explain a lot but even one small counterexample easily invalidates the whole thing. A good theory can not easily be changed to “take into account” the aberration. For example, Einstein’s theory of gravitation is a good explanation because it explains a lot, it makes counterintuitive predictions, and even one repeatable counterexample invalidates the whole thing. It can’t be easily changed to accommodate something else without invalidating everything else about it.
Theories that are hard to vary remain constant over time. They are more true and therefore more timeless. Invariable theories possess more verisimilitude (“truth-likeness” to use Popper’s term).
Like the very best possible theory, truth also cannot be varied. It is completely timeless. It was, is, and always will be true, without any change. That is Deutsch’s point.
I agree that there’s nothing new to people who have been on Overcoming Bias and Less Wrong for a few years (hence the cautionary statement at the start of the post) but I do think it’s important that we don’t forget that there are new people arriving all the time.
Not everyone would consider “the conjunction fallacy and how each detail makes your explanation less plausible” a standard point. We shouldn’t make this site inaccessible to those people. Credit where it’s due—Deutsch does a nice job of presenting this in a way that most people can understand.
I don’t think his way of explaining it is any easier for a newcomer. It doesn’t make sense unless and until you already have a firm grasp of the basis for Occam’s razor. And if you know how to justify Occam’s razor, you already understand why adding details penalizes the explanation’s probability.
Furthermore, his idea can’t be summarized as “good explanations are hard to vary”. It’s more like, “good explanations are hard to vary while preserving their predictions”.
I do appreciate that you added a summary.
I don’t think he is saying, “good explanations are hard to vary while preserving their predictions”.
As described above the statement “Everyone just acts in his own interest” very easily preserves its predictive power in a multitude of situations. Indeed, the problem with it is that the statement preserves its predictive power in too many situations! The explanation is consistent with just about whatever happens, so one cannot design a test that makes one believe that the statement is certainly false. So it is too easy to vary and hence a bad explanation.
I don’t believe this should be asserted with the level of certainty you use. His explanation makes good intuitive sense to me, and I don’t see why that must be overwhelmingly determined by preexisting understanding.
Then why didn’t you realize that the bit about “while preserving their predictions” was an essential part of Deutsch’s explanation and therefore include it in your summary of his idea?
If it was hard to see why you should have included that part, then it seems to me that Deutsch’s approach doesn’t clarify matters.
If I merely refer to his explanation, I need not say more. There is no point in arguing that my few-words summary is significantly less explanatory than the whole talk.
What about the other elements of the iceberg not explicitly mentioned, either in my summary or the talk itself? As an idea of rational methodology, Deutsch’s message is already implicit in any person’s mind, one only has to fill the gaps, if there is no ambiguity as to which idea was being discussed. Of course, it’s impossible to expect this kind of ingenuity of the audience, but then the question of which details are essential and which are not is moot.
Of course you can’t recap the whole talk, but you can describe the critical part. Very few good ideas actually need a full fifteen minutes to express. Look at what my post did for the talk: explained Deutsch’s phrasing of Occam’s razor. That’s the critical part people were wondering about, and the explanation I gave sufficed to tell those people whether the talk would be worth their time.
It seems summarizing is a lost art.
I disagree that what you singled out here is critical.
The title of this top-level post is: “David Deutsch: A new way to explain explanation”.
That makes Deutsch’s explanation of explanation critical.
Which is why I slogged through the video to get to it and save everyone else the time.
I think that ‘whilst preserving the predictions’ was assumed. Otherwise what’s the constraint that’s making things hard?
Perhaps it’s clearer when written more explicitly though.
It is assumed; it’s just not clear to someone who’s told that that’s Deutsch’s idea.
And I certainly wasn’t alone in not realizing what “hard to vary” means here; Vladimir_Nesov already had a +5 comment with the term that attempted to summarize the lecture, but my comment with the fuller explanation still got modded up to 4 and some thanks. This probably wouldn’t have happened if Nesov’s summary, using just “hard to vary”, were already clear enough.
I don’t agree with your summary.
By your own admission you haven’t watched the entire talk. That might make it difficult to provide a full review.
By reducing what Deutsch said to the conjunction fallacy you missed the different emphasis that both Vladimir and I found interesting. If the people that voted up your comment didn’t watch the talk (which seems plausible because of the negative nature of the review) then they wouldn’t appreciate the difference between what Deutsch says and what you say. Therefore they aren’t agreeing with your summary, they’re simply appreciating your effort.
I summarized what was important to LW readers. I skipped through the parts of the video that most LWers would have found uninteresting (people used to posit theories with unnecessary details called “myths”? who knew?) so I could get to Deutsch’s new explanation of explanation which amounts to “unnecessary details are bad” (which are equivalent to “easy-to-vary” aspects).
Yes, you may have found it interesting. It still would have been nice to know the basic form of Deutsch’s point before blowing ~15 minutes listening to boring stuff just to get to something that can be restated in a few sentences.
(Modding my appreciated summary down sure helps your argument though.)
I welcome anyone else to blow 20 minutes of their life to confirm my summary.
I hadn’t realised that you were taking the karma ratings as indicative of agreement. I didn’t vote it down before because I have tended only to use my downvote on stupid or thoughtless comments—not valid comments that disagree with what I think.
Once it became clear that you thought that the votes weren’t just appreciating effort but were signalling agreement it would have been dishonest not to vote it down.
I don’t think voting down indicates disagreement, nor do I believe people should use mere disagreement as a reason to vote down. My point was that you can artificially increase the merit of your point by voting down my summary so as to make it look less appreciated.
This video is not really about the conjunction fallacy—it’s more about Occam’s razor—but frankly, it presents a version of the razor that’s rather muddled.
IMO, traditional presentations of the razor are of much better quality. This video seems like a very bad introduction to the topic to me. It misses the idea of compressing theories as far as the data will allow, and instead promotes a rather dumbed down “hard to vary” → good, “easy to vary” → bad philosophy.
Thanks for the pocket summary. That saves me 20 minutes I can spend watching some other TED talk.
My pleasure :-)
Read the comments to this TED talk, and try not to kill yourself in despair.
Deutsch’s fact: The truth consists of hard-to-vary assertions about reality, i.e. good explanations.
The way I see the truth: It is elegantly fragile, untouchable, at the peak of all ideas, paramount, excellent, and most importantly, it is simplistically and effortlessly beautiful.
What did you get out of this talk apart from the importance of the (Bayesian) Occam’s razor?
Sorry, I can’t resist. But it looks like one of those cases of boiling everything down into one simple rule. Not necessarily wrong, but suspicious.