A couple of days ago, prompted by several recent posts by Owen_Richardson, I checked out the book “Theory of Instruction” (Engelmann and Carnine, 1982) from my university library and promised to read it this weekend and write a post about Direct Instruction. This is that post.
Thank you for following through with this! It’s super awesome of you to take this on, then actually follow up and do what you wanted, in the timeframe you planned. This post was very good at concisely saying what it is.
Just to check that I understood it, Direct Instruction is about presenting a sequence of examples of what does and doesn’t fit a concept, geared towards making sure that the most common false ideas are falsified, and then testing with similar ideas to check for retention/comprehension?
The examples don’t have to be binary ones, those are just the easiest ones to describe and the most common. If you were teaching addition, your examples would be (more or less) addition problems, but they would still follow the same rules, with some modifications (for instance, negative examples don’t really make sense in the context of “2+3=?”).
But basically you have the right idea. Another thing I didn’t touch on in the above post is that the testing examples seem to serve a teaching role, as well. I’ve even seen example sequences in which all negative examples are left until the testing, though I haven’t read carefully enough to be able to say when one is supposed to do this.
But basically you have the right idea. Another thing I didn’t touch on in the above post is that the testing examples seem to serve a teaching role, as well.
I’ve even seen example sequences in which all negative examples are left until the testing, though I haven’t read carefully enough to be able to say when one is supposed to do this.
Okay, so looking into it further, this sometimes happens when teaching a “noun” concept (that is, a basic concept with multiple defining qualities and possibly a fuzzy boundary). The text has this to say about the matter:
Because very small differences in examples are usually not shown in a noun sequence, the sequence is usually fairly short, consisting of enough positive examples to show the range of variation and those negatives that might be confused with the positives.
There are multiple examples of noun sequences:
Learning the concept of a “truck”, distinguishing from already-known concepts of “car”, “bus”, and “train”. Three truck examples are given, then testing begins; no negative examples.
Learning the letter “b”, distinguishing from several already-known letters. Only two examples (“b” in two fonts) are given before the test examples, because the concept has a narrow range; no negative examples.
Identifying a “black oak” leaf, distinguishing it from other leaves. A “white oak” leaf is also shown in the test examples, because the two look very similar.
Learning the category “vehicle”, when various types of vehicles are known. It has negative examples of “swing” and “lawnmower” to demonstrate boundary cases, before the testing examples begin.
Ah, yup. Also heard about this effect with spaced repetition.
I’m surprised I didn’t make the connection between Direct Instruction and spaced repetition earlier. A lot of the theory of DI easily translates to making better spaced repetition tests.
Learning the letter “b”, distinguishing from several already-known letters. Only two examples (“b” in two fonts) are given before the test examples, because the concept has a narrow range; no negative examples.
I think it would be good to include also non-examples of “d”, “p” and “q”.
Generally, I think that any explanation should include non-examples, to show the boundaries of the concept. Otherwise you did not disprove the hypothesis that “anything is a valid example”.
My intuition about DI is that you give a few examples and non-examples such that an Occam’s razor will lead student to the correct explanation. Or in other words, “faultless communication” is one where the correct interpretation of teacher’s words has lower (preferably: much lower) Kolmogorov complexity than any incorrect interpretation.
One of the rules for nouns is that the negative examples you use (in the whole sequence, including testing) are ones the learner already knows. In this case, I think that, because there is such a narrow range of variation in letters, they felt like the already-known “d”, “p”, and “q” could be saved for the test examples.
I personally think it wouldn’t hurt to mention them before the testing examples, too, and this seems like something open to interpretation.
Just to check that I understood it, Direct Instruction is about presenting a sequence of examples of what does and doesn’t fit a concept, geared towards making sure that the most common false ideas are falsified, and then testing with similar ideas to check for retention/comprehension?
Seems like it. Now I understand why Richardson was comparing it to Zendo.
Or the 2-4-6 game ‘reversed’, yes. Before Misha’s post, I was actually going to try and straighten out the confusion over “logically faultless communication” by showing how it would apply to the 2-4-6 game ‘reversed’ as an example. I might still, depending.
Although the thing is that I’d say the best way to communicate something like ‘2-4-6’ isn’t as one of the simpler concepts in the hierarchy, but as a ‘cognitive routine’, which is made of a chain (possibly branching) of various simpler concepts that have already been taught.
As Misha said:
For instance, teaching integration by substitution might first involve a simple sequence of examples about identifying when the method is appropriate, then a sequence about choosing the correct substitution, before actually teaching students to solve an integration problem using the method.
(Which doesn’t even get into the hidden complexity of all parts ‘black boxed’ together in the sentence as assumed already taught)
Of course it is easier to explain “logically faultless communication” by showing how it applies to more basic concepts, than to complex concepts that are made up of many of those basic concepts connected together.
The problem is that when you just show the very basic concepts as the AthabascaU module on DI does, people say stuff like this:
And, more generally, all examples given may be used for teaching categorization of objects. How do you teach algorithms (such as multiplication)? How do you teach history and geography? How do you teach calculus? How do you teach scientific method? Not every knowledge can be reduced to questions of form “does X have property Y” taught by presenting series of objects which either are or aren’t Y. In the whole presentation there was not a single practically applicable example. Children don’t need to go to school to learn what “is longer than” or “not horizontally aligned” means.
(Prase, in comment on first DI post.)
I anticipated this, and had tried to avoid it by injecting a little excitement, like: ‘Hey y’all, here’s something extremely valuable but complex and non-obvious. It will seem confusing and/or trivial at first, but it really is valuable!’
And in actual fact, looking back, that probably did help, because I got plenty of people going, ‘What the hell are you talking about? Give us some meat!’ rather than just, ‘Huh. Whatever.’
I’m sure there was a much better way possible of achieving the same goal, but what were the chances of me ever finding it without any feedback on actual attempts?
And in actual fact, looking back, that probably did help, because I got plenty of people going, ‘What the hell are you talking about? Give us some meat!’ rather than just, ‘Huh. Whatever.’
I’m sure there was a much better way possible of achieving the same goal, but what were the chances of me ever finding it without any feedback on actual attempts?
Editors. You are not the first person I’ve seen who could really use an editor on their material.
Editors are feedback. The only place I was gonna find suitable editors was LW. Rather than PMing people, I posted it to the section called “discussion”.
For heaven’s sake, what else was I supposed to do?! :P
A very specific kind of feedback, with understood social roles and norms of interaction; it is a ‘kind’ of person, which you can solicit for. If you cannot think how else to do it because PMing random LWers (a perfectly valid way! Just ask politely and take no for an answer.), there are many other ways. Just off the top of my head:
Fanfiction has an ancient tradition of ‘beta’ readers, and well-developed mechanisms for soliciting beta readers.
General and literary writers have an even more ancient tradition of workshops and reviewing, which has migrated online; I seriously considered submitting my own material to one of the most active online editing communities, Critters Workshop.
University students (aren’t you one?) have free access to their fellow students (I helped out one such friend extensively with his writing), and more importantly, they have unlimited access to ‘writing centers’ or other such establishments on campus, whose job is helping students out with editing and reviewing their reports, essays, etc. (I’ve talked to a few people who work in them; they complain about how few students ever turn to them. The ESL students sometimes make heavy use of them, but no one else.)
And this isn’t even thinking outside the box a little; for example, I bet you could find plenty of editors/reviewers if you posted a Bitcoin advertisement.
(Also, little in your original post was LW-specific. Any intelligent college grad, for example, could offer good feedback on that. It’s not like you were discussing the finer details of UDT or something.)
No, I’m on a full time internship as an elementary teacher. The theory I’m studying by myself.
I’m surrounded by people who know how to deliver DI programs, how to do superb classroom management, etc, but not by anyone who could read Theory of Instruction in one weekend and write a report on it (How many posts of lorem ipsum would Misha have to make for me to upvote for the karma system to accurately reflect the props he deserves :P).
I’m not sure that advice is very good when the consequences of failure apparently made you contemplate suicide.
And when your post is sufficiently bad that a single editor could have diagnosed most of the problems, it’s a little contemptuous of peoples’ time to—in effect—run it simultaneously past dozens/hundreds of editors.
I wasn’t contemplating suicide per se, I was just wondering seriously whether the reason I wasn’t was essentially a form of akrasia.
And I have major past issues. I have an intellectual belief that I will one day be able to share true mutual understanding, love, and trust with other people. Relationships that make life worthwhile.
However, due to some traumatic experiences with people I thought I had that with just completely disappearing, even though I can see in retrospect how to discriminate such unreliable people from the real deal, it takes a lot of mental pumping for me to keep up the corresponding emotional belief.
So I certainly wasn’t being “contemptuous of peoples’ time” on purpose, and given that my original post had 20 upvotes at one time, I don’t think the community feels that way either.
If you understand where I’m coming from now and have changed your mind that I did make the right choice, please tell me that.
So I certainly wasn’t being “contemptuous of peoples’ time” on purpose, and given that my original post had 20 upvotes at one time, I don’t think the community feels that way either.
I stand by my comments; DI is interesting enough that even a badly written post is still a net benefit which could get 20 net votes. However, it would have been much better all around—for you and the readers both—if you had gotten someone to edit it, in any of the 3 or 5 ways I’ve outlined. There’s no reason a well-done DI article couldn’t be promoted to the main page, eg.
(In a related issue, I am also bothered that you apparently could not think of any way to get editors. You should work on that.)
I… this is one of those issues that if I am in the wrong, I will have to take a break and apply some more intense techniques for getting around my own defensiveness than I usually need to use.
But I really honestly feel no “small note of discord” in my mind that should make me expect to find that I am wrong.
At any rate, since it’s over and done with now, what say the both of us just put the issue far in the back of our minds to allow any potentially useful new thoughts to crystallize by themselves, and refocus our attention on the future of what we need to write about DI?
But I really honestly feel no “small note of discord” in my mind that should make me expect to find that I am wrong.
So you don’t care that most of the reaction to your article was about how it was written? You don’t care about how much time you’ve spent discussing it with me alone? (Or how much time I’ve spent, hoping that future material will be better?) You don’t care about how much the impact was muted because of all that? You don’t care about what you’ve learned about the value of clear writing? You don’t care about building a reputation as a guy who knows about something interesting but can’t write for beans? You don’t care about your apparent ignorance of editing done either by yourself or another, or how to get it, or that you were ignorant of being ignorant, or that you might be generally miscalibrated about your competence? You don’t care about sending the message that you don’t care about all the foregoing?
I’m not asking for a large note of discord, but I definitely think there should be a small note there somewhere.
refocus our attention on the future of what we need to write about DI?
I’d rather discuss you. DI is just one topic, and hopefully just the first of many topics you might discuss here. Someone else will sooner or later pick up the DI baton, but if you ignore any lessons to be learned here, when will you learn them? Sooner would be better than later.
After rereading your last comment here, I just wanted to make clear: I do care very much.
Thank you making an excellent, explicit, compressed list of everything I did wrong. (...Where else than LW would that be obviously non-sarcastic? :P)
It is very valuable and I will be using it to improve. If I had a printer, I’d print it out and put it by my computer. (As it is I’ll just have to save it to a file I use a lot.)
I’m going to the effort of telling you this because, due to the value of the comment, I want to encourage similar feedback from you in the future.
...And if I had never made the attempt, I never would have gotten that feedback from you either. :P
Yes, I realize that you can’t just say, “Well, it’s all right, cuz I learned a valuable lesson!” and then keep doing the same dumb thing, but… whenever I have done a dumb thing, it’s cuz I haven’t learned that lesson yet! So long as you’re not twisting that into an excuse to keep doing the dumb thing, or to avoid trying to learn faster not to do different dumb things in the future, you’ve got it right, right?
So, I’ve begun writing a new post, “A dry presentation of some empirical evidence on DI’s effectiveness”. (An attempt to replace that intended function of my original post with as high-quality a replacement as Misha’s post was for the intended function of the ‘theory sketch’ section.)
KPier very kindly offered to help me with editing, so I sent her the first seven-ish paragraphs I had written. She found one change to recommend, somewhat ambivalent herself over which way was best. I wasn’t sure either, and found myself wondering what I’d decide in the end.
Then I started wondering about what differences in responses there might be between a post where she made all the final decisions, and a post where where I did.
And then I thought… double-blind experiment! (Woot woot, raise the empirical roof. :P)
Here’s my idea:
I finish writing the post, get the ‘her final’ and ‘my final’ versions, and then make a post linking to both versions and explaining the experiment.
I’ll just label them version A and version B (flip a coin to avoid any weird bias I may have on As and Bs, not that I’d anticipate much), and ask the reader to follow one or the other (by flipping a coin to avoid any weird bias they may have; Mostly just to make sure the sample sizes for each version are equalized.)
Then people record their impression and give me their feedback (without directly quoting the text), and I have to try and discriminate which readers got which.
Does that sound like a neat idea? If it works well, it seems like it might even end up being worth creating an automated system for setting up and running such experiments (without all the coin flipping and link following), for people to use with appropriate posts.
Luke did such a test recently. It’s probably useful for feedback (right now, his two version are at 20 and 3 karma), but really annoying for commenters. I would recommend getting some beta testers instead (I volunteer). Even a small sample of readers should be able to catch most relevant problems.
Thanks! I did think it sounded annoying for commenter, and I don’t want to try the general audience’s patience much further at this point. Hence why I’m just asking a few people what they think of it in the comments.
Being able to calibrate myself objectively is an extremely attractive idea, though.
I appreciate your willingness to have an in-depth discussion of this topic with me, and if I had infinite time, I would gladly take you up on it. But since I don’t, I’m sorry, but I’m going to have to bow out of discussing the subject of me in order to have more time for the subject of DI.
I have already learned lots of things I think I could apply to better accomplishing similar goals in the future. And I don’t anticipate having to introduce another such wide, deep, complex topic as DI from scratch again.
Again, thank you, and I hope to see more of you in the DI discussion.
You know, it’s also possible to learn a totally different lesson here: We should be gentle with people, because they are often much more vulnerable than we assume. I would argue that this lesson is much more generally applicable than “Be more scared of failure.”
Please see “I wasn’t contemplating suicide per se”. I knew in advance that I would decide to keep fighting, as I always do. It is actually a technique I use to cheer myself up, rather like being underwater, and dipping down just a bit so that you can kick off the ground in order to spring back to the surface.
I certainly know I’m very careful thanks to my own experiences to avoid causing unnecessary pain to others, and in fact to try very hard to make people happier.
But I am not fragile. I hurt, but I never break.
I thanked people for their harsh criticism, remember.
Thank you for following through with this! It’s super awesome of you to take this on, then actually follow up and do what you wanted, in the timeframe you planned. This post was very good at concisely saying what it is.
Just to check that I understood it, Direct Instruction is about presenting a sequence of examples of what does and doesn’t fit a concept, geared towards making sure that the most common false ideas are falsified, and then testing with similar ideas to check for retention/comprehension?
The examples don’t have to be binary ones, those are just the easiest ones to describe and the most common. If you were teaching addition, your examples would be (more or less) addition problems, but they would still follow the same rules, with some modifications (for instance, negative examples don’t really make sense in the context of “2+3=?”).
But basically you have the right idea. Another thing I didn’t touch on in the above post is that the testing examples seem to serve a teaching role, as well. I’ve even seen example sequences in which all negative examples are left until the testing, though I haven’t read carefully enough to be able to say when one is supposed to do this.
Ah, yup. Also heard about this effect with spaced repetition.
That seems like an interesting case.
Okay, so looking into it further, this sometimes happens when teaching a “noun” concept (that is, a basic concept with multiple defining qualities and possibly a fuzzy boundary). The text has this to say about the matter:
There are multiple examples of noun sequences:
Learning the concept of a “truck”, distinguishing from already-known concepts of “car”, “bus”, and “train”. Three truck examples are given, then testing begins; no negative examples.
Learning the letter “b”, distinguishing from several already-known letters. Only two examples (“b” in two fonts) are given before the test examples, because the concept has a narrow range; no negative examples.
Identifying a “black oak” leaf, distinguishing it from other leaves. A “white oak” leaf is also shown in the test examples, because the two look very similar.
Learning the category “vehicle”, when various types of vehicles are known. It has negative examples of “swing” and “lawnmower” to demonstrate boundary cases, before the testing examples begin.
I’m surprised I didn’t make the connection between Direct Instruction and spaced repetition earlier. A lot of the theory of DI easily translates to making better spaced repetition tests.
I think it would be good to include also non-examples of “d”, “p” and “q”.
Generally, I think that any explanation should include non-examples, to show the boundaries of the concept. Otherwise you did not disprove the hypothesis that “anything is a valid example”.
My intuition about DI is that you give a few examples and non-examples such that an Occam’s razor will lead student to the correct explanation. Or in other words, “faultless communication” is one where the correct interpretation of teacher’s words has lower (preferably: much lower) Kolmogorov complexity than any incorrect interpretation.
One of the rules for nouns is that the negative examples you use (in the whole sequence, including testing) are ones the learner already knows. In this case, I think that, because there is such a narrow range of variation in letters, they felt like the already-known “d”, “p”, and “q” could be saved for the test examples.
I personally think it wouldn’t hurt to mention them before the testing examples, too, and this seems like something open to interpretation.
Seems like it. Now I understand why Richardson was comparing it to Zendo.
Or the 2-4-6 game ‘reversed’, yes. Before Misha’s post, I was actually going to try and straighten out the confusion over “logically faultless communication” by showing how it would apply to the 2-4-6 game ‘reversed’ as an example. I might still, depending.
Although the thing is that I’d say the best way to communicate something like ‘2-4-6’ isn’t as one of the simpler concepts in the hierarchy, but as a ‘cognitive routine’, which is made of a chain (possibly branching) of various simpler concepts that have already been taught.
As Misha said:
(Which doesn’t even get into the hidden complexity of all parts ‘black boxed’ together in the sentence as assumed already taught)
Of course it is easier to explain “logically faultless communication” by showing how it applies to more basic concepts, than to complex concepts that are made up of many of those basic concepts connected together.
The problem is that when you just show the very basic concepts as the AthabascaU module on DI does, people say stuff like this:
(Prase, in comment on first DI post.)
I anticipated this, and had tried to avoid it by injecting a little excitement, like: ‘Hey y’all, here’s something extremely valuable but complex and non-obvious. It will seem confusing and/or trivial at first, but it really is valuable!’
And in actual fact, looking back, that probably did help, because I got plenty of people going, ‘What the hell are you talking about? Give us some meat!’ rather than just, ‘Huh. Whatever.’
I’m sure there was a much better way possible of achieving the same goal, but what were the chances of me ever finding it without any feedback on actual attempts?
Editors. You are not the first person I’ve seen who could really use an editor on their material.
Editors are feedback. The only place I was gonna find suitable editors was LW. Rather than PMing people, I posted it to the section called “discussion”.
For heaven’s sake, what else was I supposed to do?! :P
A very specific kind of feedback, with understood social roles and norms of interaction; it is a ‘kind’ of person, which you can solicit for. If you cannot think how else to do it because PMing random LWers (a perfectly valid way! Just ask politely and take no for an answer.), there are many other ways. Just off the top of my head:
Fanfiction has an ancient tradition of ‘beta’ readers, and well-developed mechanisms for soliciting beta readers.
General and literary writers have an even more ancient tradition of workshops and reviewing, which has migrated online; I seriously considered submitting my own material to one of the most active online editing communities, Critters Workshop.
University students (aren’t you one?) have free access to their fellow students (I helped out one such friend extensively with his writing), and more importantly, they have unlimited access to ‘writing centers’ or other such establishments on campus, whose job is helping students out with editing and reviewing their reports, essays, etc. (I’ve talked to a few people who work in them; they complain about how few students ever turn to them. The ESL students sometimes make heavy use of them, but no one else.)
And this isn’t even thinking outside the box a little; for example, I bet you could find plenty of editors/reviewers if you posted a Bitcoin advertisement.
(Also, little in your original post was LW-specific. Any intelligent college grad, for example, could offer good feedback on that. It’s not like you were discussing the finer details of UDT or something.)
No, I’m on a full time internship as an elementary teacher. The theory I’m studying by myself.
I’m surrounded by people who know how to deliver DI programs, how to do superb classroom management, etc, but not by anyone who could read Theory of Instruction in one weekend and write a report on it (How many posts of lorem ipsum would Misha have to make for me to upvote for the karma system to accurately reflect the props he deserves :P).
But yeah, “Just Try It”, right?
And it’s the LW community that I have a strong emotional desire to get involved in, and the eventual intended audience anyway.
I’m not sure that advice is very good when the consequences of failure apparently made you contemplate suicide.
And when your post is sufficiently bad that a single editor could have diagnosed most of the problems, it’s a little contemptuous of peoples’ time to—in effect—run it simultaneously past dozens/hundreds of editors.
I wasn’t contemplating suicide per se, I was just wondering seriously whether the reason I wasn’t was essentially a form of akrasia.
And I have major past issues. I have an intellectual belief that I will one day be able to share true mutual understanding, love, and trust with other people. Relationships that make life worthwhile.
However, due to some traumatic experiences with people I thought I had that with just completely disappearing, even though I can see in retrospect how to discriminate such unreliable people from the real deal, it takes a lot of mental pumping for me to keep up the corresponding emotional belief.
LessWrong is my HOPE. A community that makes me feel that I could expect to find that understanding, love, and trust.
So I certainly wasn’t being “contemptuous of peoples’ time” on purpose, and given that my original post had 20 upvotes at one time, I don’t think the community feels that way either.
If you understand where I’m coming from now and have changed your mind that I did make the right choice, please tell me that.
I stand by my comments; DI is interesting enough that even a badly written post is still a net benefit which could get 20 net votes. However, it would have been much better all around—for you and the readers both—if you had gotten someone to edit it, in any of the 3 or 5 ways I’ve outlined. There’s no reason a well-done DI article couldn’t be promoted to the main page, eg.
(In a related issue, I am also bothered that you apparently could not think of any way to get editors. You should work on that.)
I… this is one of those issues that if I am in the wrong, I will have to take a break and apply some more intense techniques for getting around my own defensiveness than I usually need to use.
But I really honestly feel no “small note of discord” in my mind that should make me expect to find that I am wrong.
At any rate, since it’s over and done with now, what say the both of us just put the issue far in the back of our minds to allow any potentially useful new thoughts to crystallize by themselves, and refocus our attention on the future of what we need to write about DI?
So you don’t care that most of the reaction to your article was about how it was written? You don’t care about how much time you’ve spent discussing it with me alone? (Or how much time I’ve spent, hoping that future material will be better?) You don’t care about how much the impact was muted because of all that? You don’t care about what you’ve learned about the value of clear writing? You don’t care about building a reputation as a guy who knows about something interesting but can’t write for beans? You don’t care about your apparent ignorance of editing done either by yourself or another, or how to get it, or that you were ignorant of being ignorant, or that you might be generally miscalibrated about your competence? You don’t care about sending the message that you don’t care about all the foregoing?
I’m not asking for a large note of discord, but I definitely think there should be a small note there somewhere.
I’d rather discuss you. DI is just one topic, and hopefully just the first of many topics you might discuss here. Someone else will sooner or later pick up the DI baton, but if you ignore any lessons to be learned here, when will you learn them? Sooner would be better than later.
After rereading your last comment here, I just wanted to make clear: I do care very much.
Thank you making an excellent, explicit, compressed list of everything I did wrong. (...Where else than LW would that be obviously non-sarcastic? :P)
It is very valuable and I will be using it to improve. If I had a printer, I’d print it out and put it by my computer. (As it is I’ll just have to save it to a file I use a lot.)
I’m going to the effort of telling you this because, due to the value of the comment, I want to encourage similar feedback from you in the future.
...And if I had never made the attempt, I never would have gotten that feedback from you either. :P
Yes, I realize that you can’t just say, “Well, it’s all right, cuz I learned a valuable lesson!” and then keep doing the same dumb thing, but… whenever I have done a dumb thing, it’s cuz I haven’t learned that lesson yet! So long as you’re not twisting that into an excuse to keep doing the dumb thing, or to avoid trying to learn faster not to do different dumb things in the future, you’ve got it right, right?
So I still think “Just Try It” was good advice.
I’ll be sure to criticize you in the future, then.
Precisely. To err is human, to persevere is of the devil, or however it goes.
Most excellent Gwern!
I have a proposition!
So, I’ve begun writing a new post, “A dry presentation of some empirical evidence on DI’s effectiveness”. (An attempt to replace that intended function of my original post with as high-quality a replacement as Misha’s post was for the intended function of the ‘theory sketch’ section.)
KPier very kindly offered to help me with editing, so I sent her the first seven-ish paragraphs I had written. She found one change to recommend, somewhat ambivalent herself over which way was best. I wasn’t sure either, and found myself wondering what I’d decide in the end.
Then I started wondering about what differences in responses there might be between a post where she made all the final decisions, and a post where where I did.
And then I thought… double-blind experiment! (Woot woot, raise the empirical roof. :P)
Here’s my idea:
I finish writing the post, get the ‘her final’ and ‘my final’ versions, and then make a post linking to both versions and explaining the experiment.
I’ll just label them version A and version B (flip a coin to avoid any weird bias I may have on As and Bs, not that I’d anticipate much), and ask the reader to follow one or the other (by flipping a coin to avoid any weird bias they may have; Mostly just to make sure the sample sizes for each version are equalized.)
Then people record their impression and give me their feedback (without directly quoting the text), and I have to try and discriminate which readers got which.
Does that sound like a neat idea? If it works well, it seems like it might even end up being worth creating an automated system for setting up and running such experiments (without all the coin flipping and link following), for people to use with appropriate posts.
Luke did such a test recently. It’s probably useful for feedback (right now, his two version are at 20 and 3 karma), but really annoying for commenters. I would recommend getting some beta testers instead (I volunteer). Even a small sample of readers should be able to catch most relevant problems.
Thanks! I did think it sounded annoying for commenter, and I don’t want to try the general audience’s patience much further at this point. Hence why I’m just asking a few people what they think of it in the comments.
Being able to calibrate myself objectively is an extremely attractive idea, though.
It’s been done before, but not often, so I infer it doesn’t work well. Possibly this is just due to clumsy implementation.
And I’ll be sure to strive to make your job much smaller. =]
I appreciate your willingness to have an in-depth discussion of this topic with me, and if I had infinite time, I would gladly take you up on it. But since I don’t, I’m sorry, but I’m going to have to bow out of discussing the subject of me in order to have more time for the subject of DI.
I have already learned lots of things I think I could apply to better accomplishing similar goals in the future. And I don’t anticipate having to introduce another such wide, deep, complex topic as DI from scratch again.
Again, thank you, and I hope to see more of you in the DI discussion.
You know, it’s also possible to learn a totally different lesson here: We should be gentle with people, because they are often much more vulnerable than we assume. I would argue that this lesson is much more generally applicable than “Be more scared of failure.”
It’s far easier to get the person with vivid memories of contemplating suicide to be more cautious than to get everyone in general to be more gentle.
Well, that’s certainly a good point, but I wasn’t talking to everyone in general, either.
Please see “I wasn’t contemplating suicide per se”. I knew in advance that I would decide to keep fighting, as I always do. It is actually a technique I use to cheer myself up, rather like being underwater, and dipping down just a bit so that you can kick off the ground in order to spring back to the surface.
I certainly know I’m very careful thanks to my own experiences to avoid causing unnecessary pain to others, and in fact to try very hard to make people happier.
But I am not fragile. I hurt, but I never break.
I thanked people for their harsh criticism, remember.
.