Some people have expressed some doubts about this story, and because it is anonymous, we can’t verify it directly. I would like to use this opportunity to explore our models of the school system, and especially the difference between the models of insiders and outsiders.
This poll asks a pair of questions. The first question is about how the story fits your model of educational system. The second question is whether you are an “school system insider”. That means whether you ever had a full-time job or a part-time job related to the school system; whether you were a teacher or a director of a school, an employee of a Department of Education, a school inspector, or an employee of a company working for education as much as the author of this article. (It is not enough to be a student, a janitor in a school, or an employee of a company which sold one product to schools but sells most of its products elsewhere.)
You move from ‘very difficult to believe’ to ‘seems like a very exaggerated version of true events,’ which is almost the same thing but from that you directly jump to ‘This story is credible’ without any middle ground. I am sorry but this seems like a really poor questionnaire design.
Once the poll is made, the answers cannot be edited. The difference was supposed to mean approximately a) I don’t believe this could ever happen, b) I believe it can happen exceptionally, but not all the time as the author claims, c) I believe this can be the way system works.
In other words, the second option is like: “I believe that with so many grant proposals, once in a while a crazy thing passes unnoticed; but I don’t believe that it happens all the time, not even half of the time—you have probably seen one or two bad cases, and now you exaggerate to make your case more appealing”.
I found myself wanting to say “I think this sort of grant proposal thing happens maybe 25% of the time, but not all the time, the way the post implies.”
I also wished there was some kind of gradation for “school insider/outsider”. I’m an outsider, but I talk a lot with a friend who teaches full time. I showed this article to her and she said “yes, yes, yes. This is basically how it is.”
I actually DO still assign substantial probability to this being a hoax, despite it matching my understanding—we know the this is sockpuppet account, created ostensibly for NDA anonymity. But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
Spreading false data as a “test of rationality” would be actively harmful. But I can imagine people misunderstanding that.
Rationality is a method of working with the data you have. You should update on evidence correctly, instead of updating incorrectly. You should be able to recognize that this specific piece of evidence contradicts the model based on all other evidence, which makes this specific piece of evidence suspicious. But also should estimate your degree of certainty in a given model.
It is proper to say “I defy the data” when one’s model is based on a lot of reliable evidence. Saying it more often would be overconfidence, not rationality.
I agree that this would be the most likely course of action if the essay is a hoax, but I think it would still risk being harmful overall, since retractions generally don’t result in an appropriate corresponding decrease in confidence in the material that was originally presented. I’d expect Less Wrong members in general to be better at reducing their confidence in a retracted claim than most people, but better is not necessarily good enough.
But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
Which people? Stylometry might us allow to work out whether someone’s writting style matches this post. Anonymity is hard.
Yes, we could probably de-anonymize OP; I have some passing familiarity with stylometrics, so I considered trying that myself. I decided not to because so far no one has produced any smoking guns that this is false (school districts vary massively in quality across the USA & I have already mentioned an existing and far more shocking instance of school failure/corruption), if it is true then I approve of whistleblowing and have no interest in attacking the OP*, de-anonymizing probably would not set a good precedent, and if it were false—well, I do not disapprove of red team tests of LW (and would be hypocritical to disapprove) and so far this seems to be limited to LW.
* Similarly, it’s been suggested to me by a few people that it would be an interesting project to try to de-anonymize Satoshi Nakamoto or La Griffe du Lion. I am not sure I could, and even if I could, I would choose not to since I either approve of their work or find their material interesting.
If this post ever looked like it was both false and not a limited-scale test, like it was something else (an entrapment of an off-site person? an attempt to discredit LW entirely with a Sokal-style attack?), then I might change my mind. But so far, that does not seem to be the case since I see nothing indicating this post has been picked up by the Drudge Report, Hacker News, Fox News, Breitbart, etc.
Yeah, I also generally consider that posts like this have around 5% chances of being a hoax followed by a gloating “they swallowed it” update (here or somewhere else), though this post doesn’t have any huge red flags (there doesn’t seem to be any huge gloating potential, I mean it’s basically just venting).
So, you are asking me to condition on belief that the author of this post isn’t a troll who crafted this story in such a way to be especially appealing to the LW community. Am I right? If I am, you probably should edit your original post to make your idea more clear.
b) I believe it can happen exceptionally, but not all the time as the author claims
That sounds very different from “This story seems like a very exaggerated version of true events”. One is about frequency (how often do things in the author’s intended reference class occur?) while the other is about severity (how bad are the events that actually happened to the author?)
My thoughts: having seen lots of people trying to write things and failing miserably—my father the professor repeatedly finds that large numbers of students don’t follow directions when writing lab reports—I’m not surprised by a claim that people would do stupid things when writing grant applications and then not understand what the problem is when it was pointed out to them.
That means whether you ever had a full-time job or a part-time job related to the school system; whether you were a teacher or a director of a school, an employee of a Department of Education, a school inspector, or an employee of a company working for education as much as the author of this article
I’m a school system insider by this standard. This makes the survey rather broken because I know very little about schools in your country. I recommend correcting your language.
The educational systems seems to me similar enough in different countries. When I read stories of teachers in Britain or USA, of course there are differences, but the similarities are also obvious. Some of the stories could have as well happened in my class.
The biggest differences are: How the school is organized. (Are teachers in a union? Who appoints the director? Is there a management layer between the director and the teachers?) Which minorities underperform, and what political consequences does it have. (Black students? Romani students? Are there ethnical quotas? Affirmative action? How often are teachers accused of racism; what is the typical reason and typical consequences?)
Another significant difference is the level of violence at school—but this varies also within the country, and changes during time. (How often and how severly do students attack each other? Do students attack teachers? Do parents attack teachers? What are the consequences for the agressor?)
And here are the similarities: Bureaucracy. Decisions made by people who don’t have a clue, and often have zero educational experience. Supervision and assessment according to unintelligible or actively harmful criteria. Pseudoscience, and aversion to measuring outcomes. (Tests are bad. Teachers shouldn’t explain, but entertain. If any recommended technique doesn’t work, it is always the teacher’s fault, never a problem with the technique. Knowledge is a lost purpose, the true goal of the school system is to make students happy.) People making big money selling pseudoscience to schools. Random minor changes in school system to make voters see that politicians care about their children. Textbooks containing nonsense. Parents treating teachers as babysitters.
And here are the similarities: Bureaucracy. Decisions made by people who don’t have a clue, and often have zero educational experience. Supervision and assessment according to unintelligible or actively harmful criteria. Pseudoscience, and aversion to measuring outcomes. (Tests are bad. Teachers shouldn’t explain, but entertain. If any recommended technique doesn’t work, it is always the teacher’s fault, never a problem with the technique. Knowledge is a lost purpose, the true goal of the school system is to make students happy.) People making big money selling pseudoscience to schools. Random minor changes in school system to make voters see that politicians care about their children. Textbooks containing nonsense. Parents treating teachers as babysitters.
I mostly agree (strongly) with this. However the “Tests are bad” part in particular doesn’t seem to be completely general. More testing and measurement seems to be the direction things have been going here.
The educational systems seems to me similar enough in different countries.
Ok, I’ve answered the survey adopting this assumption. I chose “Very difficult to believe, school system insider”. But note that I would also have found the prom segregation difficult to believe if not for the somewhat credible sources so discount the results as appropriate.
I have taught in a school and subbed in others. I had very little say about who could come into my classroom, though I was only a first year teacher. (Quit after 1 year). I don’t have any reason to believe education beaurocracy is generally bright or honest, but can’t say I’ve seen that kind of thing in the grant process, possibly just because I wasn’t involved in it.
I voted earlier then came back to this post to see how it was going. How do I view the results? I tried voting again, but it won’t let me, and I’m sorry, I can’t guess how to view the poll results anymore.
Some people have expressed some doubts about this story, and because it is anonymous, we can’t verify it directly. I would like to use this opportunity to explore our models of the school system, and especially the difference between the models of insiders and outsiders.
This poll asks a pair of questions. The first question is about how the story fits your model of educational system. The second question is whether you are an “school system insider”. That means whether you ever had a full-time job or a part-time job related to the school system; whether you were a teacher or a director of a school, an employee of a Department of Education, a school inspector, or an employee of a company working for education as much as the author of this article. (It is not enough to be a student, a janitor in a school, or an employee of a company which sold one product to schools but sells most of its products elsewhere.)
[pollid:425]
You move from ‘very difficult to believe’ to ‘seems like a very exaggerated version of true events,’ which is almost the same thing but from that you directly jump to ‘This story is credible’ without any middle ground. I am sorry but this seems like a really poor questionnaire design.
Once the poll is made, the answers cannot be edited. The difference was supposed to mean approximately a) I don’t believe this could ever happen, b) I believe it can happen exceptionally, but not all the time as the author claims, c) I believe this can be the way system works.
In other words, the second option is like: “I believe that with so many grant proposals, once in a while a crazy thing passes unnoticed; but I don’t believe that it happens all the time, not even half of the time—you have probably seen one or two bad cases, and now you exaggerate to make your case more appealing”.
I found myself wanting to say “I think this sort of grant proposal thing happens maybe 25% of the time, but not all the time, the way the post implies.”
I also wished there was some kind of gradation for “school insider/outsider”. I’m an outsider, but I talk a lot with a friend who teaches full time. I showed this article to her and she said “yes, yes, yes. This is basically how it is.”
I actually DO still assign substantial probability to this being a hoax, despite it matching my understanding—we know the this is sockpuppet account, created ostensibly for NDA anonymity. But can think of some people here who might have created this explicitly as a test of rationality, who are sort of annoyed that the politics involved here get less scrutiny and want to demonstrate that.
Spreading false data as a “test of rationality” would be actively harmful. But I can imagine people misunderstanding that.
Rationality is a method of working with the data you have. You should update on evidence correctly, instead of updating incorrectly. You should be able to recognize that this specific piece of evidence contradicts the model based on all other evidence, which makes this specific piece of evidence suspicious. But also should estimate your degree of certainty in a given model.
It is proper to say “I defy the data” when one’s model is based on a lot of reliable evidence. Saying it more often would be overconfidence, not rationality.
I was assuming that if it were a hoax, they’d let people know in another few days with a gloating update.
I agree that this would be the most likely course of action if the essay is a hoax, but I think it would still risk being harmful overall, since retractions generally don’t result in an appropriate corresponding decrease in confidence in the material that was originally presented. I’d expect Less Wrong members in general to be better at reducing their confidence in a retracted claim than most people, but better is not necessarily good enough.
Which people? Stylometry might us allow to work out whether someone’s writting style matches this post. Anonymity is hard.
Yes, we could probably de-anonymize OP; I have some passing familiarity with stylometrics, so I considered trying that myself. I decided not to because so far no one has produced any smoking guns that this is false (school districts vary massively in quality across the USA & I have already mentioned an existing and far more shocking instance of school failure/corruption), if it is true then I approve of whistleblowing and have no interest in attacking the OP*, de-anonymizing probably would not set a good precedent, and if it were false—well, I do not disapprove of red team tests of LW (and would be hypocritical to disapprove) and so far this seems to be limited to LW.
* Similarly, it’s been suggested to me by a few people that it would be an interesting project to try to de-anonymize Satoshi Nakamoto or La Griffe du Lion. I am not sure I could, and even if I could, I would choose not to since I either approve of their work or find their material interesting.
If this post ever looked like it was both false and not a limited-scale test, like it was something else (an entrapment of an off-site person? an attempt to discredit LW entirely with a Sokal-style attack?), then I might change my mind. But so far, that does not seem to be the case since I see nothing indicating this post has been picked up by the Drudge Report, Hacker News, Fox News, Breitbart, etc.
Yeah, I also generally consider that posts like this have around 5% chances of being a hoax followed by a gloating “they swallowed it” update (here or somewhere else), though this post doesn’t have any huge red flags (there doesn’t seem to be any huge gloating potential, I mean it’s basically just venting).
So, you are asking me to condition on belief that the author of this post isn’t a troll who crafted this story in such a way to be especially appealing to the LW community. Am I right? If I am, you probably should edit your original post to make your idea more clear.
That sounds very different from “This story seems like a very exaggerated version of true events”. One is about frequency (how often do things in the author’s intended reference class occur?) while the other is about severity (how bad are the events that actually happened to the author?)
My thoughts: having seen lots of people trying to write things and failing miserably—my father the professor repeatedly finds that large numbers of students don’t follow directions when writing lab reports—I’m not surprised by a claim that people would do stupid things when writing grant applications and then not understand what the problem is when it was pointed out to them.
I’m a school system insider by this standard. This makes the survey rather broken because I know very little about schools in your country. I recommend correcting your language.
Whose country? Viliam_Bur’s country is most probably not the same country as the OP author’s.
In that case the survey makes even less sense to me.
The educational systems seems to me similar enough in different countries. When I read stories of teachers in Britain or USA, of course there are differences, but the similarities are also obvious. Some of the stories could have as well happened in my class.
The biggest differences are: How the school is organized. (Are teachers in a union? Who appoints the director? Is there a management layer between the director and the teachers?) Which minorities underperform, and what political consequences does it have. (Black students? Romani students? Are there ethnical quotas? Affirmative action? How often are teachers accused of racism; what is the typical reason and typical consequences?)
Another significant difference is the level of violence at school—but this varies also within the country, and changes during time. (How often and how severly do students attack each other? Do students attack teachers? Do parents attack teachers? What are the consequences for the agressor?)
And here are the similarities: Bureaucracy. Decisions made by people who don’t have a clue, and often have zero educational experience. Supervision and assessment according to unintelligible or actively harmful criteria. Pseudoscience, and aversion to measuring outcomes. (Tests are bad. Teachers shouldn’t explain, but entertain. If any recommended technique doesn’t work, it is always the teacher’s fault, never a problem with the technique. Knowledge is a lost purpose, the true goal of the school system is to make students happy.) People making big money selling pseudoscience to schools. Random minor changes in school system to make voters see that politicians care about their children. Textbooks containing nonsense. Parents treating teachers as babysitters.
I mostly agree (strongly) with this. However the “Tests are bad” part in particular doesn’t seem to be completely general. More testing and measurement seems to be the direction things have been going here.
I read “Tests are bad” as “The tests do not accurately measure”.
Ok, I’ve answered the survey adopting this assumption. I chose “Very difficult to believe, school system insider”. But note that I would also have found the prom segregation difficult to believe if not for the somewhat credible sources so discount the results as appropriate.
I have taught in a school and subbed in others. I had very little say about who could come into my classroom, though I was only a first year teacher. (Quit after 1 year). I don’t have any reason to believe education beaurocracy is generally bright or honest, but can’t say I’ve seen that kind of thing in the grant process, possibly just because I wasn’t involved in it.
I voted earlier then came back to this post to see how it was going. How do I view the results? I tried voting again, but it won’t let me, and I’m sorry, I can’t guess how to view the poll results anymore.
EDIT: I figured it out, on the off chance anybody else wonders, hit the chain-link looking icon at the bottom of the poll to go to a page where you see results (at least if you have voted). That link points to: http://lesswrong.com/r/discussion/lw/h5z/problems_in_education/8qa5