Khan Academy doesn’t come close to executing the dependency tree thing thoroughly enough. There are a lot of concepts that are unexplained, and Khan Academy doesn’t test to see what you know before teaching you. When I refer to the dependency tree, I mean for it to have concepts on a much smaller scale than the typical calculus, precalculus, algebra, trig etc. An even smaller scale than this. I apologize for not being able to effectively communicate the type of scale I’m trying to refer to.
There are definitely some parallels with Sal’s ideas though. For example, mastery based learning, learning from a web app, learning at your own pace and at your own convenience, and having smaller certificates. I’m not trying to “steal” his ideas or anything. I just see a huge bottleneck that is preventing progress (the absence of dependency tree driven education), and thought it’d be worth writing about.
Your inability to effectively communicate the kind of scale you had in mind might be because we don’t actually know what the prerequisite tree looks like at that scale, and just finding out would be a major step forward for educational science.
It’s also not clear that any unique dependency tree even exists at such a fine-grained scale, because there exist multiple different ways of accomplishing a goal. Feynman talked about this, when recounting the episode where he learned that he counted time by mental speech, whereas his friend counted time by watching a visualization of a tape with flowing numbers:
By that experience Tukey and I discovered that what goes on in different people’s heads when they think they’re doing the same thing—something as simple as counting—is different for different people. And we discovered that you can externally and objectively test how the brain works: you don’t have to ask a person how he counts and rely on his own observations of himself; instead, you observe what he can and can’t do while he counts. The test is absolute. There’s no way to beat it; no way to fake it.
It’s natural to explain an idea in terms of what you already have in your head. Concepts are piled on top of each other; this idea is taught in terms of that idea, and that idea is taught in terms of another idea, which comes from counting, which can be so different for different people! I often think about that, especially when I’m teaching some esoteric technique such as integrating Bessel functions. When I see equations, I see the letters in colors—I don’t know why. As I’m talking, I see vague pictures of Bessel functions from Jahnke and Emde’s book, with light-tan j’s, slightly violet-bluish n’s, and dark brown x’s flying around. And I wonder what the hell it must look like to the students.
Similarly, komponisto had an explanation of how he understood speed, and how that made it hard for him to understand some kinds of problems, because his mental concept of speed was a nonstandard one.
So there’s unlikely to exist any single dependency tree, because at such a fine-grained level, it’s likely that different people will have multiple different ways of comprehending any single concept. If my comprehension of “counting” is some specific variant X, then that might make some follow-up concept so obvious as to not even be worth explaining, while making some other follow-up concept hard to understand even with explanation. Whereas if my comprehension of counting was based on variant Y instead, then the pattern might be reversed.
I think you can see this assumption in constructivist teaching methods, which start out from the assumption that everyone filters their learning through and bases it on their existing knowledge, which will be different for everyone. Thus, rather than there being a pre-existing curriculum with a clear dependency tree that the teacher tries to transfer into the heads of the learners, the methods instead
rely on some form of guided discovery where the teacher avoids most direct instruction and attempts to lead the student through questions and activities to discover, discuss, appreciate, and verbalize the new knowledge.
Thus, instead of the teacher trying to chart out the exact fine-grained dependency tree, the students are encouraged to work on the problems in such a way that they themselves find the ways of learning the topic that are suitable for their pre-existing knowledge and ways of conceptualizing the world. You do still have some dependency tree, of course, but it’s at a higher level than the one you seem to be talking about.
It’s also not clear that any unique dependency tree even exists at such a fine-grained scale, because there exist multiple different ways of accomplishing a goal.
Maybe, but consider how often in todays system a teacher or textbook doesn’t explain something that they should. Even if the dependency tree wasn’t perfect, it’d still be much much better than what we have today.
Possibly. But you were selling this as one of the most important things to focus on if we wish to improve education. Given the immense effort that this would require, it’s massively unobvious that this would be the most cost-effective thing to concentrate on right now.
I would put stuff like improving motivation by making the content of the teaching more relevant to people’s daily lives, breaking down the artificial distinctions between subjects, applying the educational and motivational techniques that the makers of commercial videogames have developed, etc. etc. as things that were more useful to focus on at this moment. If the students are motivated enough, they can work out the problems themselves, even if not all of the inferential steps were completely explained.
It is one of the most important things and extremely cost effective
Given enough data, the dependency tree can be inferred implicity from and we can also learn to accurately identify when a student doesn’t understand a concept. I’m actually working on this exact problem right now
adamzerner said that he wanted a really fine-grained dependency map, at an even lower level than this. Are you also talking about such a level of fine-grainedness? I agree that dependency maps on the level of that map are likely to be cost-effective, it’s the more fine-grained ones that I was skeptical of.
If you are talking about an even lower level than the one displayed in that map, I’d love to hear more about it.
It is possible to detect correlations between questions (or even individual steps) and to implicitly determine skills at an even more fine grained level than that—given sufficient data. We aren’t aiming to get that specific right at the moment—indeed, we mightn’t have collected enough data yet to do that—but there’s no reason why we shouldn’t be able to do that
Having tagged data is extremely useful for designing these kinds of algorithms, but we would only require a proportion of the data to be tagged. So it wouldn’t be anywhere near as costly as you expect
Ohh, the Big Data approach. That makes sense, and also explains why it would be possible to do cost-effectively now when previous attempts haven’t gotten very far. Not sure why I didn’t think of that. Okay, formally retracting my skepticism.
Excuse me if I’m misunderstanding your ideas here, but isn’t this, almost bullet for bullet, exactly what Khan academy is doing?
Khan Academy doesn’t come close to executing the dependency tree thing thoroughly enough. There are a lot of concepts that are unexplained, and Khan Academy doesn’t test to see what you know before teaching you. When I refer to the dependency tree, I mean for it to have concepts on a much smaller scale than the typical calculus, precalculus, algebra, trig etc. An even smaller scale than this. I apologize for not being able to effectively communicate the type of scale I’m trying to refer to.
There are definitely some parallels with Sal’s ideas though. For example, mastery based learning, learning from a web app, learning at your own pace and at your own convenience, and having smaller certificates. I’m not trying to “steal” his ideas or anything. I just see a huge bottleneck that is preventing progress (the absence of dependency tree driven education), and thought it’d be worth writing about.
Your inability to effectively communicate the kind of scale you had in mind might be because we don’t actually know what the prerequisite tree looks like at that scale, and just finding out would be a major step forward for educational science.
It’s also not clear that any unique dependency tree even exists at such a fine-grained scale, because there exist multiple different ways of accomplishing a goal. Feynman talked about this, when recounting the episode where he learned that he counted time by mental speech, whereas his friend counted time by watching a visualization of a tape with flowing numbers:
Similarly, komponisto had an explanation of how he understood speed, and how that made it hard for him to understand some kinds of problems, because his mental concept of speed was a nonstandard one.
So there’s unlikely to exist any single dependency tree, because at such a fine-grained level, it’s likely that different people will have multiple different ways of comprehending any single concept. If my comprehension of “counting” is some specific variant X, then that might make some follow-up concept so obvious as to not even be worth explaining, while making some other follow-up concept hard to understand even with explanation. Whereas if my comprehension of counting was based on variant Y instead, then the pattern might be reversed.
I think you can see this assumption in constructivist teaching methods, which start out from the assumption that everyone filters their learning through and bases it on their existing knowledge, which will be different for everyone. Thus, rather than there being a pre-existing curriculum with a clear dependency tree that the teacher tries to transfer into the heads of the learners, the methods instead
Thus, instead of the teacher trying to chart out the exact fine-grained dependency tree, the students are encouraged to work on the problems in such a way that they themselves find the ways of learning the topic that are suitable for their pre-existing knowledge and ways of conceptualizing the world. You do still have some dependency tree, of course, but it’s at a higher level than the one you seem to be talking about.
Maybe, but consider how often in todays system a teacher or textbook doesn’t explain something that they should. Even if the dependency tree wasn’t perfect, it’d still be much much better than what we have today.
Possibly. But you were selling this as one of the most important things to focus on if we wish to improve education. Given the immense effort that this would require, it’s massively unobvious that this would be the most cost-effective thing to concentrate on right now.
I would put stuff like improving motivation by making the content of the teaching more relevant to people’s daily lives, breaking down the artificial distinctions between subjects, applying the educational and motivational techniques that the makers of commercial videogames have developed, etc. etc. as things that were more useful to focus on at this moment. If the students are motivated enough, they can work out the problems themselves, even if not all of the inferential steps were completely explained.
It is one of the most important things and extremely cost effective
Given enough data, the dependency tree can be inferred implicity from and we can also learn to accurately identify when a student doesn’t understand a concept. I’m actually working on this exact problem right now
adamzerner said that he wanted a really fine-grained dependency map, at an even lower level than this. Are you also talking about such a level of fine-grainedness? I agree that dependency maps on the level of that map are likely to be cost-effective, it’s the more fine-grained ones that I was skeptical of.
If you are talking about an even lower level than the one displayed in that map, I’d love to hear more about it.
It is possible to detect correlations between questions (or even individual steps) and to implicitly determine skills at an even more fine grained level than that—given sufficient data. We aren’t aiming to get that specific right at the moment—indeed, we mightn’t have collected enough data yet to do that—but there’s no reason why we shouldn’t be able to do that
Having tagged data is extremely useful for designing these kinds of algorithms, but we would only require a proportion of the data to be tagged. So it wouldn’t be anywhere near as costly as you expect
Ohh, the Big Data approach. That makes sense, and also explains why it would be possible to do cost-effectively now when previous attempts haven’t gotten very far. Not sure why I didn’t think of that. Okay, formally retracting my skepticism.
One of the first things that the paper mentioned was that Feynman died on Feb 15th.
Boo, Death.