Second of all, we still don’t really know what “best” means, and it’s entirely possible different methods are best for different people in complex ways.
And there’s a worse confounding factor, which is that people tend to interpret instructions in terms of whatever prior model they have. (That’s actually why I object so strenuously to a couple of aspects of your “oath” model—they’re not so much intrinsically harmful, as harmful to people with certain prior models.)
Testing the distinction between your method and mine would require pretty stringent behavioral control of subjects in a large experiment, because you’d need to validate that the subject actually considered each situation and consequence. (Writing those things out is a good way to verify it, which is why I think your success was actually a side-effect of the thinking you had to do in order to design and write your oaths.)
However, if you just grab a bunch of volunteers and tell them to do either your version or mine of that process, I predict that a substantial number will not actually follow the directions, and will simply tell themselves they’ve already thought it through enough after considering maybe 1 or 2 situations, and then proceed to do whatever it is they already do to initiate change effects, sprinkled with a bit of flavor from whatever method they’re supposed to be testing.
This is a major confounding factor in testing any cognitive behavior model, be it a self-help technique, time management system, or anything else. People tend to process virtually all new inputs through whatever mental strategies they already have, and lop off the parts that don’t fit.
Estimating actual treatment effects is possible but not practical. Is that a fair summary of the parent?
I’m just saying it’s hard, and that informal means won’t work very well. Well-designed experiments in psychology tend to be designed to trick people into doing or thinking the thing that’s being tested, in order to avoid some of these effects.
And there’s a worse confounding factor, which is that people tend to interpret instructions in terms of whatever prior model they have. (That’s actually why I object so strenuously to a couple of aspects of your “oath” model—they’re not so much intrinsically harmful, as harmful to people with certain prior models.)
Testing the distinction between your method and mine would require pretty stringent behavioral control of subjects in a large experiment, because you’d need to validate that the subject actually considered each situation and consequence. (Writing those things out is a good way to verify it, which is why I think your success was actually a side-effect of the thinking you had to do in order to design and write your oaths.)
However, if you just grab a bunch of volunteers and tell them to do either your version or mine of that process, I predict that a substantial number will not actually follow the directions, and will simply tell themselves they’ve already thought it through enough after considering maybe 1 or 2 situations, and then proceed to do whatever it is they already do to initiate change effects, sprinkled with a bit of flavor from whatever method they’re supposed to be testing.
This is a major confounding factor in testing any cognitive behavior model, be it a self-help technique, time management system, or anything else. People tend to process virtually all new inputs through whatever mental strategies they already have, and lop off the parts that don’t fit.
All we can feasibly get is the intent-to-treat effect. Estimating actual treatment effects is possible but not practical.
Is that a fair summary of the parent?
I’m just saying it’s hard, and that informal means won’t work very well. Well-designed experiments in psychology tend to be designed to trick people into doing or thinking the thing that’s being tested, in order to avoid some of these effects.
I mean not practical for the LW community in the way MichaelVassar would like to see happen.