I have only skimmed your post, but now feel motivated to leave feedback as requested. It is possible that some of my objections are misplaced, addressed somewhere in the depths of this article that my eyes glazedly passed over. In fact, my first complaint is that:
it is too long. LW tolerates long articles under limited circumstances and this doesn’t meet any of them (you’re not an established poster, don’t have fifty footnotes with sources, don’t apologize off the bat for length, and have missed many obvious opportunities for compression/excision). You should have made it much shorter (500 words about what the hell Direct Instruction consists of) or much much shorter (a two-sentence blurb with a link to more information).
It’s sales-y. Full of applause lights (counted five instances of the string “rational” in your text). You claim that your intent is to pique interest, but that is not done by saying “This thing is interesting! This thing is interesting!” repeatedly in the local idiom.
It is badly structured. Rambles all over the place. If you laid out the contents of your article in conceptspace and made me walk from point to point in the order you present them, my feet would get tired and I would become dizzy. You have definitely not convinced me that you have learned a secret of how to teach things, on a meta as well as object level.
It makes you look like a crank. If DI needs this much fluff and meandering and enthusiastic pitching, it’s probably not interesting. Oops.
In fact, the only reason I am bothering to think about this article ever again, having successfully scrolled all the way down to the unnecessary signature, is that you do repeatedly ask for feedback. If you’re sincere about that: I invite you to post, as a reply to this comment, a 1-3 sentence description of what DI is, plus one sentence about whatever evidence (beyond your enthusiasm about it) which exists for its splendidness. (Last sentence but not the first 1-3 can be/consist primarily of linkage.)
I’m in strong agreement, with the one addition that it would be interesting to see an explanation of DI using DI principles, and then an explanation of how the principles were used to shape the explanation.
I felt like I was slogging through this article to glean the relevant bits, which turned out to be the axioms and principles and the history and study evidence for the efficacy of the system.
To the author:
Keep those relevant bits. Expand on them with examples and ramifications. Cut everything else. Aggressively.
Assume that you’re talking to people who already care about the effectiveness of instructional methods; who are already basically aware of issues such as neurotypicality/neurodiversity even; and who are aware of the difference between saying you care about a goal, and actually acting to maximize that goal.
Assume that we already have knowledge and skills that we desperately want to teach, because we highly value ensuring that humanity retains that knowledge and those skills. Give us tools to accomplish our existing goals.
I’ll have to think about a good “1-3 sentence description of what DI is” (in terms of what you can do with it, or in terms of its internal structure, or...?) while I’m at work, but as for evidence, if I said:
“Look at those two graphs from Project Follow-Through, because the meta-analysis says this stark difference between DI and other models of education keeps showing up in experiments comparing them.”
Direct Instruction is an educational theory which extensively used experiments during its creation, and seeks to explain educational concepts in a sensible and efficient way. The amount of experience individual teachers can acquire pales in comparison to the amount of experience curriculum-builders can acquire, which DI takes advantage of by giving teachers heavily researched, and thus effective, scripts and practices to follow. Experiments dramatically verify the superiority of DI over other instruction methods.
In three sentences, you can’t explain anything about the method besides its inputs and its outputs, which is what other people are interested in. If they’ve got a use for your method, then they’ll start asking about its moving parts.
That explains DI’s origins, but doesn’t say anything about how it looks like and how it is different from other approaches. (Why it is called Direct Instruction, after all?)
Right- that’s because I don’t know anything about what it looks like and how it’s different from other approaches. (I haven’t dug any deeper than this article and wikipedia).
Agreed, the main problem is that you don’ seem to explain what the theory is until deep into the text. the section ‘what it looks like in practice’ doesn’t actually tell us anything other than that scripts are maybe involved, and that teachers don’t like it (personally i found the tone unnecessarily harsh there).
The next sections with graphs seem persuasive, but again at this point we don’t know what it is they are proposing, or where this data is coming from. Perhaps better would be just to link to the study with a quick (one or two sentence) summary of its findings.
Finally we get an explanation of what it is we have been talking about all this time in ‘a quick sketch of the basic theory.’ First paragraph is why we should like it, then an explanation of how you are explaining it.
As far as I can tell what its saying is:
Use examples to illustrate the concept, they should be similar in only the important ways, and counter examples that are different in relevant ways.
Information should be arranged in a logical way,
All I get from the example is that its better to teach via rules than via memorisation.
At the risk of being harsh or flippant is that it? That doesn’t seem like a massive educational innovation to me (though I should say I don’t know much about the US system which seems the example here).
If there is something more to it (which I would hope given your enthusiasm) you need to make it very clear what that is. All the stuff about evidence, teachers and examples is unhelpful until we know that.
that teachers don’t like it (personally i found the tone unnecessarily harsh there).
From my understanding of the alternate education literature, pretty much everyone is bitter at teachers as a group and has pretty good reason to be. The harshness didn’t even register until now because I see it so often.
Is the thesis that the majority of teachers and education employees are engaged in an active conspiracy to prevent the introduction of new teaching methods? I find that difficult to believe without substantial evidence. (It sets off my warning lights by similarities to claims made by ‘alternative’ medicine advocates about the biased scienetifc establishment and general conspiracy theories).
Even if a conspiracy existed, given the incentives individual teachers/schools/local governments have to improve performance of students why wouldn’t they defect and reap the benefits?
Is the thesis that the majority of teachers and education employees are engaged in an active conspiracy to prevent the introduction of new teaching methods?
I can’t speak in sweeping generality here, but I can speak from my own experience. (I’m finishing a dissertation in educational research and bump into this pretty frequently.)
Teachers who have been “in the trenches” for any appreciable length of time (say, 5+ years) have developed some sense of what is necessary from a pragmatic point of view. They’re usually extremely well-intentioned toward the children, and often for that reason they resist a lot of the suggestions brought to them from educational research. There’s this sense that education researchers are too ivory-tower to know how things really happen in a classroom. There are exceptions, but they’re actually quite rare.
However, not all education researchers are naive about this issue. I know of many who were teachers in public schools for a decade or so before they moved on to research to do something about the mess they personally encountered. They’re able to build face-to-face rapport more readily with teachers. But that’s simply not good enough to convince teachers as a collective to try something new, even if the pragmatics of how to do it are spelled out in excruciating detail. They seem to resist change purely because it’s change, especially if it interferes with their personal emotional impressions of what it means to teach (which, unfortunately, were instilled more by accidental impressions than by training or rational scrutiny).
In addition to this challenge, a painfully large portion of education researchers don’t apply reasonable scientific methods. The majority of articles I’ve encountered in this field amount to philosophical commentary, which is often based on spotty evidence that to me would constitute preliminary research. So surprise surprise, a lot of the reform suggestions being proposed are saturated in cognitive biases that no one seems to be at all aware of let alone make any effort to account for. And, thus, most “well-researched” reform suggestions really don’t work in classrooms! So on the rare occasions that teachers are willing to try something radically new based on a new theory (or are required to by fiat), in most cases it doesn’t make much of a difference at best.
And this is ignoring the fact that researchers frequently forget about (or never consider!) the inferential distance teachers have to traverse to even understand what they’re supposed to do let alone do it well enough to teach. The “New Math” of the 1960s was a spectacular failure both because the theory behind it was psychologically and neurologically bankrupt and also because the teachers weren’t able to implement the original idea anyway.
So no, I don’t think it’s a conspiracy. I think it’s more that teachers are tired of having people try to mechanize their profession as though it didn’t require any skill to interact with children, especially since the vast majority of attempts to do so that they’ve encountered failed dramatically—often because the reform wouldn’t have worked anyway, but often because it was implemented very poorly due to undertraining the teachers in the first place.
Of course it’s not a “conspiracy” per se. It’s just yer standard lost purposes at work, in a field where the population really doesn’t have a strong grasp of the basic philosophy of science, leaning very much to “romanticism” rather than “enlightenment”, I mean.
Thanks for your help. It’s more… well, look: We at LessWrong are very familiar with the idea that if you really want to pin some hypothesis down, there’s a certain minimum amount of work you have to do. You can’t just magically jump there without certain bits of information, and processing the bits together in the right way. And most of the work goes into just locating the general area of the correct hypothesis.
Okay, that’s obvious. We get that. The big thing DI does (in the stimulus-locus analysis) is turn that sideways and apply it to teaching.
There are certain bits of information without which the naive student can not just magically figure out what we’re trying to communicate, so we need to sequence the introduction of those bits of information, make each logically unambiguous, and prompt the proper processing of the bits together in a manageable context that lets us ensure each step has gotten across properly before moving on to the next bit. The teaching communication should be designed to at least strongly imply the correct conclusion as early as possible.
Thank you, I’m glad to know I’m at least making some progress (I sweated over my first attempt for ages and it ended up terrible, but then just a little time of feedback and back and forth discussion seems to really be tightening up my understanding of what I need to communicate and how! That’s probably a highly generalizable principle :P - actually reminds me of some “just make an attempt already if failure is low cost!” post I saw some months back in main… can’t find it right now, but maybe you remember the one I mean. [Edit: it was “Just Try It”)
Anyway, is this also helpful? (Even ‘unutterably’ so? :P)
I have only skimmed your post, but now feel motivated to leave feedback as requested. It is possible that some of my objections are misplaced, addressed somewhere in the depths of this article that my eyes glazedly passed over. In fact, my first complaint is that:
it is too long. LW tolerates long articles under limited circumstances and this doesn’t meet any of them (you’re not an established poster, don’t have fifty footnotes with sources, don’t apologize off the bat for length, and have missed many obvious opportunities for compression/excision). You should have made it much shorter (500 words about what the hell Direct Instruction consists of) or much much shorter (a two-sentence blurb with a link to more information).
It’s sales-y. Full of applause lights (counted five instances of the string “rational” in your text). You claim that your intent is to pique interest, but that is not done by saying “This thing is interesting! This thing is interesting!” repeatedly in the local idiom.
It is badly structured. Rambles all over the place. If you laid out the contents of your article in conceptspace and made me walk from point to point in the order you present them, my feet would get tired and I would become dizzy. You have definitely not convinced me that you have learned a secret of how to teach things, on a meta as well as object level.
It makes you look like a crank. If DI needs this much fluff and meandering and enthusiastic pitching, it’s probably not interesting. Oops.
In fact, the only reason I am bothering to think about this article ever again, having successfully scrolled all the way down to the unnecessary signature, is that you do repeatedly ask for feedback. If you’re sincere about that: I invite you to post, as a reply to this comment, a 1-3 sentence description of what DI is, plus one sentence about whatever evidence (beyond your enthusiasm about it) which exists for its splendidness. (Last sentence but not the first 1-3 can be/consist primarily of linkage.)
I’m in strong agreement, with the one addition that it would be interesting to see an explanation of DI using DI principles, and then an explanation of how the principles were used to shape the explanation.
I’d like to see this too. The example given of DI was way too contrived.
Yes.
I felt like I was slogging through this article to glean the relevant bits, which turned out to be the axioms and principles and the history and study evidence for the efficacy of the system.
To the author:
Keep those relevant bits. Expand on them with examples and ramifications. Cut everything else. Aggressively.
Assume that you’re talking to people who already care about the effectiveness of instructional methods; who are already basically aware of issues such as neurotypicality/neurodiversity even; and who are aware of the difference between saying you care about a goal, and actually acting to maximize that goal.
Assume that we already have knowledge and skills that we desperately want to teach, because we highly value ensuring that humanity retains that knowledge and those skills. Give us tools to accomplish our existing goals.
Ouch ouch ouch, but thank you for helping me.
I’ll have to think about a good “1-3 sentence description of what DI is” (in terms of what you can do with it, or in terms of its internal structure, or...?) while I’m at work, but as for evidence, if I said:
“Look at those two graphs from Project Follow-Through, because the meta-analysis says this stark difference between DI and other models of education keeps showing up in experiments comparing them.”
Is that at all useful?
How about:
Direct Instruction is an educational theory which extensively used experiments during its creation, and seeks to explain educational concepts in a sensible and efficient way. The amount of experience individual teachers can acquire pales in comparison to the amount of experience curriculum-builders can acquire, which DI takes advantage of by giving teachers heavily researched, and thus effective, scripts and practices to follow. Experiments dramatically verify the superiority of DI over other instruction methods.
In three sentences, you can’t explain anything about the method besides its inputs and its outputs, which is what other people are interested in. If they’ve got a use for your method, then they’ll start asking about its moving parts.
That explains DI’s origins, but doesn’t say anything about how it looks like and how it is different from other approaches. (Why it is called Direct Instruction, after all?)
Right- that’s because I don’t know anything about what it looks like and how it’s different from other approaches. (I haven’t dug any deeper than this article and wikipedia).
Agreed, the main problem is that you don’ seem to explain what the theory is until deep into the text. the section ‘what it looks like in practice’ doesn’t actually tell us anything other than that scripts are maybe involved, and that teachers don’t like it (personally i found the tone unnecessarily harsh there).
The next sections with graphs seem persuasive, but again at this point we don’t know what it is they are proposing, or where this data is coming from. Perhaps better would be just to link to the study with a quick (one or two sentence) summary of its findings.
Finally we get an explanation of what it is we have been talking about all this time in ‘a quick sketch of the basic theory.’ First paragraph is why we should like it, then an explanation of how you are explaining it.
As far as I can tell what its saying is:
All I get from the example is that its better to teach via rules than via memorisation.
At the risk of being harsh or flippant is that it? That doesn’t seem like a massive educational innovation to me (though I should say I don’t know much about the US system which seems the example here).
If there is something more to it (which I would hope given your enthusiasm) you need to make it very clear what that is. All the stuff about evidence, teachers and examples is unhelpful until we know that.
[Hope this helps, apologies if it sounds harsh.]
From my understanding of the alternate education literature, pretty much everyone is bitter at teachers as a group and has pretty good reason to be. The harshness didn’t even register until now because I see it so often.
Is the thesis that the majority of teachers and education employees are engaged in an active conspiracy to prevent the introduction of new teaching methods? I find that difficult to believe without substantial evidence. (It sets off my warning lights by similarities to claims made by ‘alternative’ medicine advocates about the biased scienetifc establishment and general conspiracy theories).
Even if a conspiracy existed, given the incentives individual teachers/schools/local governments have to improve performance of students why wouldn’t they defect and reap the benefits?
I can’t speak in sweeping generality here, but I can speak from my own experience. (I’m finishing a dissertation in educational research and bump into this pretty frequently.)
Teachers who have been “in the trenches” for any appreciable length of time (say, 5+ years) have developed some sense of what is necessary from a pragmatic point of view. They’re usually extremely well-intentioned toward the children, and often for that reason they resist a lot of the suggestions brought to them from educational research. There’s this sense that education researchers are too ivory-tower to know how things really happen in a classroom. There are exceptions, but they’re actually quite rare.
However, not all education researchers are naive about this issue. I know of many who were teachers in public schools for a decade or so before they moved on to research to do something about the mess they personally encountered. They’re able to build face-to-face rapport more readily with teachers. But that’s simply not good enough to convince teachers as a collective to try something new, even if the pragmatics of how to do it are spelled out in excruciating detail. They seem to resist change purely because it’s change, especially if it interferes with their personal emotional impressions of what it means to teach (which, unfortunately, were instilled more by accidental impressions than by training or rational scrutiny).
In addition to this challenge, a painfully large portion of education researchers don’t apply reasonable scientific methods. The majority of articles I’ve encountered in this field amount to philosophical commentary, which is often based on spotty evidence that to me would constitute preliminary research. So surprise surprise, a lot of the reform suggestions being proposed are saturated in cognitive biases that no one seems to be at all aware of let alone make any effort to account for. And, thus, most “well-researched” reform suggestions really don’t work in classrooms! So on the rare occasions that teachers are willing to try something radically new based on a new theory (or are required to by fiat), in most cases it doesn’t make much of a difference at best.
And this is ignoring the fact that researchers frequently forget about (or never consider!) the inferential distance teachers have to traverse to even understand what they’re supposed to do let alone do it well enough to teach. The “New Math” of the 1960s was a spectacular failure both because the theory behind it was psychologically and neurologically bankrupt and also because the teachers weren’t able to implement the original idea anyway.
So no, I don’t think it’s a conspiracy. I think it’s more that teachers are tired of having people try to mechanize their profession as though it didn’t require any skill to interact with children, especially since the vast majority of attempts to do so that they’ve encountered failed dramatically—often because the reform wouldn’t have worked anyway, but often because it was implemented very poorly due to undertraining the teachers in the first place.
Of course it’s not a “conspiracy” per se. It’s just yer standard lost purposes at work, in a field where the population really doesn’t have a strong grasp of the basic philosophy of science, leaning very much to “romanticism” rather than “enlightenment”, I mean.
Does the Rosenhan experiment imply any ‘conspiracies’ in psychiatry?
Thanks for your help. It’s more… well, look: We at LessWrong are very familiar with the idea that if you really want to pin some hypothesis down, there’s a certain minimum amount of work you have to do. You can’t just magically jump there without certain bits of information, and processing the bits together in the right way. And most of the work goes into just locating the general area of the correct hypothesis.
Okay, that’s obvious. We get that. The big thing DI does (in the stimulus-locus analysis) is turn that sideways and apply it to teaching.
There are certain bits of information without which the naive student can not just magically figure out what we’re trying to communicate, so we need to sequence the introduction of those bits of information, make each logically unambiguous, and prompt the proper processing of the bits together in a manageable context that lets us ensure each step has gotten across properly before moving on to the next bit. The teaching communication should be designed to at least strongly imply the correct conclusion as early as possible.
That make any sense?
Aha!
This is a very useful and relevant explanation, which would have been unutterably more useful to read in the article itself.
Thank you, I’m glad to know I’m at least making some progress (I sweated over my first attempt for ages and it ended up terrible, but then just a little time of feedback and back and forth discussion seems to really be tightening up my understanding of what I need to communicate and how! That’s probably a highly generalizable principle :P - actually reminds me of some “just make an attempt already if failure is low cost!” post I saw some months back in main… can’t find it right now, but maybe you remember the one I mean. [Edit: it was “Just Try It”)
Anyway, is this also helpful? (Even ‘unutterably’ so? :P)