Understanding your understanding

Re­lated to: Truly Part of You, A Tech­ni­cal Ex­pla­na­tion of Tech­ni­cal Ex­pla­na­tion

Partly be­cause of LessWrong dis­cus­sions about what re­ally counts as un­der­stand­ing (some typ­i­cal ex­am­ples), I came up with a scheme to clas­sify differ­ent lev­els of un­der­stand­ing so that posters can be more pre­cise about what they mean when they claim to un­der­stand—or fail to un­der­stand—a par­tic­u­lar phe­nomenon or do­main.

Each level has a de­scrip­tion so that you know if you meet it, and tells you what to watch out for when you’re at or close to that level. I have taken the liberty of nam­ing them af­ter the LW ar­ti­cles that de­scribe what such a level is like.

Level 0: The “Guess­ing the Teacher’s Pass­word” Stage

Sum­mary: You have no un­der­stand­ing, be­cause you don’t see how any out­come is more or less likely than any other.

De­scrip­tion: This level is only in­cluded for com­par­i­son—to show some­thing that is not un­der­stand­ing. At this point, you have, a best, la­bels that other peo­ple use when de­scribing the phe­nomenon. Maybe you can even gen­er­ate the ap­pear­ance of un­der­stand­ing on the topic. How­ever, you ac­tu­ally have a max­i­mum en­tropy prob­a­bil­ity dis­tri­bu­tion. In other words, noth­ing would sur­prise you, no event is more or less likely to hap­pen, and ev­ery­thing is con­sis­tent with what you “know” about it. No ra­tio­nal­ist should count this as an un­der­stand­ing, though it may in­volve knowl­edge of the la­bels that a do­main uses.

Things to watch out for: Scien­tific-sound­ing terms in your vo­cab­u­lary that don’t cor­re­spond to an ac­tual pre­dic­tive model; your in­abil­ity to say what you ex­pect to see, and what you would be sur­prised by.

Level 1: The “Shut up and Calcu­late” Stage

Sum­mary: You can suc­cess­fully pre­dict the phe­nomenon, but see it as an in­de­pen­dent, com­part­men­tal­ized do­main.

De­scrip­tion: This is where you can pre­dict the phe­nomenon, us­ing a gen­er­a­tive model that tells you what to ex­pect. You are ca­pa­ble of be­ing sur­prised, as cer­tain ob­ser­va­tions are as­signed low prob­a­bil­ity. It may even be tremen­dously com­pli­cated, but it works.

Though low on the hi­er­ar­chy, it’s ac­tu­ally a big ac­com­plish­ment in it­self. How­ever, when you are at this stage, you see its dy­nam­ics as be­ing un­re­lated to any­thing else, be­long­ing to its own do­main, fol­low­ing its own rules. While it might have par­allels to things you do un­der­stand, you see no rea­son why the par­allel must hold, and there­fore can’t rea­son about how ex­ten­sive that re­la­tion­ship is.

Things to watch out for: Go­ing from “It just works, I don’t know what it means” to “it doesn’t mean any­thing!” Also, be­com­ing proud of your ig­no­rance of its re­la­tion­ship to the rest of the world.

Level 2: The “En­tan­gled Truths” Stage. (Alter­nate name: “Univer­sal Fire”.)

Sum­mary: Your ac­cu­rate model in this do­main has deep con­nec­tions to the rest of your mod­els (whether in­fer­en­tial or causal); in­fer­ences can flow be­tween the two.

De­scrip­tion: At this stage, your model of the phe­nomenon is also deeply con­nected to your model of ev­ery­thing else. In­stead of the phe­nomenon be­ing some­thing with its own set of rules, you see how its dy­nam­ics in­ter­face with the dy­nam­ics of ev­ery­thing else in your un­der­stand­ing. You can de­rive pa­ram­e­ters in this do­main from your knowl­edge in an­other do­main; you can ex­plain how they are re­lated.

Note the re­gres­sion here: you meet this stage when your model for the new phe­nomenon con­nects to your model for “ev­ery­thing else”. So what about the first “ev­ery­thing else” you un­der­stood (which could be called your “prim­i­tively un­der­stood” part of re­al­ity)? This would be the in­stinc­tive model of the world that you are born with: the “folk physics”, “folk psy­chol­ogy”, etc. Its ex­is­tence is re­vealed in such ex­per­i­ments as when ba­bies are con­fused by rol­ling balls that sud­denly vi­o­late the laws of physics.

This “Level 2” un­der­stand­ing there­fore ul­ti­mately con­nects ev­ery­thing back to your di­rect, raw ex­pe­riences (“qualia”) of the world, but, im­por­tantly, is not sub­or­di­nate to them – op­ti­cal illu­sions shouldn’t over­ride the stronger ev­i­dence that proves to you it’s an illu­sion.

Things to watch out for: As­sum­ing that similar be­hav­ior in differ­ent do­mains (“sur­face analo­gies”) is enough to ex­plain their re­la­tion­ship. Also, us­ing one in­ter­sec­tion be­tween mul­ti­ple do­mains as a rea­son to im­me­di­ately col­lapse them to­gether.

Level 3: The “Truly Part of You” Stage

Sum­mary: Your mod­els are such that you would re-dis­cover them, for the right rea­sons, even they were deleted from your mem­ory.

De­scrip­tion: At this stage, not only do you have good, well-con­nected mod­els of re­al­ity, but they are so well-grounded, that they “re­gen­er­ate” when “dam­aged”. That is, you weren’t merely fed these won­der­ful mod­els out­right by some other Really Smart Be­ing (though ini­tially you might have been), but rather, you also con­sis­tently use a re­li­able method for gain­ing knowl­edge, and this method would even­tu­ally stum­ble upon the same model you have now, no mat­ter how much knowl­edge is stripped away from it.

This ca­pa­bil­ity arises be­cause your high un­der­stand­ing makes much of your knowl­edge re­dun­dant: know­ing some­thing in one do­main has im­pli­ca­tions in quite dis­tant do­mains, lead­ing you to rec­og­nize what was lost – and your re­li­able meth­ods of in­fer­ence tell you what, if any­thing, you need to do to re­cover it.

This stage should be the goal of all ra­tio­nal­ists.

Things to watch out for: Hind­sight bias: you may think you would have made the same in­fer­ences at a pre­vi­ous epistemic state, but that might just be due to already know­ing the an­swers. Also, if you’re re­ally at this stage, you should have what amounts to a “foun­tain of knowl­edge” – are you learn­ing all you can from it?

In con­clu­sion: In try­ing to en­hance your own, or some­one else’s, un­der­stand­ing of a topic, I recom­mend iden­ti­fy­ing which level you both are at to see if you have some­thing to learn from each other, or are sim­ply us­ing differ­ent stan­dards.