Did some research. The claim that the proposals are poorly written leaps out at me as immediately true. Here’s a website with successful grant applications, to be used as models to write them:
First, the horrible spelling, grammar, and punctuation leap out at me immediately. Second, the claim in the post that grant proposals are written to describe what they’re doing, rather than what they’re intending to achieve, holds up, for this grant at least.
This proposal is the best-written I encountered. It describes the specific problems it intends to resolve and the specific solutions it intends to use. Unfortunately, the only evidence it introduces is the evidence that there is a problem. It doesn’t provide any evidence that its solutions work. Its stated “Method of Evaluation”, moreover, exactly mirrors the claims made in this post—it evaluates whether or not its solutions are implemented, NOT whether or not the problem is solved. (Goal #5 seems like an exception, but remember the stated problem is mental health issues.
This proposal is the best I’ve encountered. It is horribly written, however. (You can skip past the pages and pages of documentation about how exactly the money will be spent to read the goals.) The accountability section has this (this is a proposal, essentially, to buy more modern computers for students and teachers, and to hire support staff):
We will begin by taking benchmarks of our current situation with regard to number of computers per student (including the capability of
that equipment), number of teachers and students currently using the PLATO or other learning systems, number of teachers using the Web
as a training and communication tool, and student scores on the TABE and PLATO assessment tests. On a quarterly basis, we will review
computer ratios and teachers/students using PLATO. Every semester we will use questionnaires and surveys, as well as observation, of
staff to get feedback on the impact of professional development activities. Student scores will be reviewed after every semester. Results
will be tabulated and communicated to school staff, superintendents, Advisory Boards and Texans Can! staff annually. Where indicated,
adjustments in curriculum and instruction will be made to ensure that student performance continues to improve
Note that the accountability, as it pertains to this grant, is—wait for it—to make sure the grant money is spent as expected. You can change curriculum and instruction -without- the computers, remember. (I think this is a pretty sensible grant request, but the accountability measures it proposes provide no actual accountability. “Did we say what we were going to do? Yes? Then our grant was a success!”)
Setting aside the fact that I’ve seen better writing from middle school students, this is actually a decently written grant. It has specific goals, implementations, and even has accountability. (Although it does seem confused about who or what is accountable to who or what; the accountability section reads rather like the author’s understanding of accountability meant the ability of students to measure their own improvements in performance. Notably omitted is a suggestion that the program’s success/improvement rate be compared to non-program success/improvement rates.)
A well-written grant I can’t find fault with. (Except maybe its questionable notion of scientific evidence.)
So—some of the grants here definitely show symptoms of the problems indicated in the post. Some don’t. A couple of these had no business being granted. ALL of these grants were successful applications—that is, the grants were granted.
After this exercise, my position shifted from “This post is credible” to “This post exaggerates the extent of the problem to some degree, but remains a valid criticism of the grant system as it exists.”
And I tried to find a grant similar to the iPod/Makeover grant, and found this:
Okay, not exactly analogous, as it at least pertains to education. However, given the grant’s self-evaluation criteria, student scores could plummet and the project could still call itself an overall success. (Actual improvement in student abilities only accounts for a fourth of their apparently unweighted criteria.)
First, the horrible spelling, grammar, and punctuation leap out at me immediately.
I just read that grant in its entirety. I noticed one possible typo, but did not find other bad grammar or spelling.
Second, the claim in the post that grant proposals are written to describe what they’re doing, rather than what they’re intending to achieve, holds up, for this grant at least.
The are asking for a grant to get equipment, primarily computers and software, for use in teaching students. It is not really a research project. What is the outcome hoped for from a grant like that? That students will be taught using these computers. They make a feint at claiming it will raise grades or enrolement, but really if I were a science teacher, my real goal would be to get the stuff and sit students down in front of it and teach them with it. I think that is pretty accurately reflected.
I’ll look at the ipad grant, and kudos for finding the site and bringing me that much closer to real contact with the kinds of grants under discussion.
First, the horrible spelling, grammar, and punctuation leap out at me immediately.
Me too. Good thing they’re not trying to improve writing ability!
I just read that grant in its entirety. I noticed one possible typo, but did not find other bad grammar or spelling.
The VERY FIRST SENTENCE has minor punctuation issues and refers to “Excellence in Leaning (sp) Through Technology”—I refuse to believe that the original Senate bill being referred to failed to spell the word “Learning” correctly in its title. :-)
The second sentence puts a space before the colon for no apparent reason.
“The moneys this school is requesting” ⇒ should probably be “money”, though I’d accept argument to the contrary.
“With request to …” ⇒ should probably be “With RESPECT to”
“This shows community support for improvement and a move forward with the support of a technology plan.” ⇒ You can tell what the writer is trying to say, but the writer is not actually saying it; the sentence is just broken.
“Teachers will...learn ho to integrate this technology” ⇒ should be “learn HOW to integrate...”
That’s just the first page, and it’s not even ALL the issues on the first page. Fortunately, the following pages are much better than the abstract page (which was painful). The second page is missing a bunch of hyphens—that’s a problem throughout—but otherwise not too bad.
Third page: “A desired outcome of this project is an increase in tile number of students taking high level science.” ⇒ change “tile number” to “total number” and possibly change “high level” to “high-level”
“By using MBL’s, less time is required” ⇒ change “MBL’s” to “MBLs”—it’s not a possessive.
“The purchase of this equipment would be in support of Colorado economy.” has a missing article; change it to ⇒ “would support THE Colorado economy”
“accommodate this set Up.” ⇒ “setup”.
Under IMPACT: “By obtaining these funds and implementing this program more students will be able to participate in hands on leaning” ⇒ again, it should be LEARNING, not LEANING. Also it’s “hands-on”, not “hands on”
“This science lab will be in place alter the grant period is over.” ⇒ AFTER the grant period, not ALTER.
Much of this suggests a very bad writer—less than 8th-grade level—who is using a spell-checker. But there some other mistakes that seem like the document might have been electronically scanned. For instance, the budget mentions “guides for teachers arid students” ⇒ should obviously be “teachers AND students” but I can’t imagine a human writer accidentally writing “arid” for “and” and “ri” does look an awful lot like “n”.
“By using MBL’s, less time is required” ⇒ change “MBL’s” to “MBLs”—it’s not a possessive.
If you look in an old enough style guide (the current standard is as you say), it will say to use an apostrophe when you pluralize an acronym. Wikipedia agrees.
With request to the Boulder Valley School District Science Content Standards, Nederland Middle Senior High School is highly motivated to reform their Science Department.
respect, regard, or reference.
Teachers will attend training sessions to become proficient with the technology and learn ho to integrate this technology into their classroom.
how
In order to meet the Science Content Standards set forth by Boulder Valley School District, our Science Department requires hardware. software, training and curriculum.
hardware, software
With a Microcomputer Based Laboratory (MBL) students can perform sophisticated experiments, collect and manipulate data, share their findings with classmates and do in depth analysis of natural phenomena.
in–depth
Productivity in the laboratory will be increased due to computers performing the data manipulation, enabling students more time to concentrate on scientific principals and concepts.
principles
With a more hands-on approach to science, many students who lose interest in science past the graduation requirements could find science to be more relevant to their day to day lives.
day–to–day
This science lab will be in place alter the grant period is over.
after
Team Labs will receive $30,175 for science curriculum, software and probeware. In addition, they will receive $2.250 for teacher training.
$2,250
Consistently ‘moneys’ is used where ‘money’ or ‘monies’ seems correct to me; I did not count this as an error despite not following a strict style guide. Most other ‘errors’ are very reasonably scanning errors rather than writing errors; the only error that couldn’t plausibly be a scanner error would be ‘principal’ for ‘principle’.
Overall, the writing was simplistic, sentences were short and simple, and would pass a technical writing test. Presented as a model for what complexity and intelligence level of grants are approved, that is very informative. Grant proposals (apparently) should be simple, repetitive, and full of Capitalized Buzzwords that are Important to the Right People.
I actually counted the really short sentences heavily against them mentally, probably too much. Owing to the way I parse sentences, reading the grant was like listening to William Shatner at his… not quite hammiest, but pretty close.
An alternative explanation is offered here: http://lesswrong.com/lw/h5z/problems_in_education/8qqj (Specifically, that the document may have been electronically scanned; this could also account for other apparent spelling mistakes. Handwriting recognition is getting better, but is still far from perfect.)
I’ll put it this way: in the average GRE scores by intended field, education ranks below philosophy & STEM in every subtest, and various forms of education rank very low (early childhood education is, out of 50 groups, second from the bottom in 2 subtests and fifth from the bottom in the last subtest).
Average GRE is useless. Elementary teachers have far lower GRE scores than secondary school teachers, and are about average in verbal and below average in math. Secondary school content teachers are above average in verbal and average in math. However, close to half of all secondary school teachers get higher than 600 on the math section, which is more than the number of math and science teachers. While I suppose it’s possible that math and science teachers have terrible math scores and the English/history teachers are scoring those 600+ scores, I’m figuring it’s far more likely that math and science high school teachers have eminently respectable GRE scores in math, and that English/history teachers have higher than average verbal.
Anyone who claims that teachers are stupid is using propaganda instead of ETS data.
Not surprising, given my experience. Most religion majors I’ve met were relatively smart and often made fun of the more fundamentalist/evangelical types who typically were turned off by their religion classes. Religion majors seemed like philosophy-lite majors (which is consistent with the rankings).
Edit: Also, relative to Religion, econ has a bunch of poor english speakers that pull the other two categories down. (Note: the “analytical” section is/was actually a couple of very short essays)
Also, Economics is only quantitatively better than Religion.
Yes, given that economics is apparently one of the most lucrative fields around going by Caplan’s recent post on majors, it’s interesting that the econ students aren’t ranked even higher.
Another category that jumped out at me—see all the public/education/business administration near the bottom. These are the people running institutions. The world is the way it is for a reason.
I notice that I’m confused: the maximum score on the Quantitative section is 800 (at that time), and Ph.D. econ programs won’t even consider you if you’re under a 780. The quantitative exam is actually really easy for math types. When you sign up for the GRE, you get a free CD with 2 practice exams. When I took it, I took the first practice exam without studying at all and got a 760 or so on the quantitiative section (within 10 pts). After studying I got a 800 on the second practice exam and on the actual exam, I got a 790. The questions were basic algebra for the most part with a bit of calculus and basic stat at the top end and a tricky question here and there. The exam was easy—really easy. I was a math major at a tiny / terrible liberal arts school; nothing like MIT or any self respecting state school. So it seems like it should be easy for anyone with a halfway decent mathematics background.
Now you’re telling me people intending to major in econ in grad school average a 706, and people intending to major in math average a 733? That’s low. Really low relative to my expectations. I would have expected a 730 in econ and maybe a 760 in math.
Possible explanations:
1) Tons of applicants who don’t want to believe that they aren’t cut out for their field create a long tail on the low side while the high side is capped at 800.
2) Master’s programs are, in general, more lenient and there are a large number of people who only intend to go to them, creating the same sort of long tail effect as above in 1).
3) There’s way more low-tier graduate programs than I thought in both fields willing to accept the average or even below average student.
4) Weirdness in how these fields are classified (e.g. I don’t see statistics there anywhere, is that included in math?)
5) the quantitative section of the standard GRE actually doesn’t matter if you’re headed to a math or physics program (someone in that field care to comment?). Note: the quantitative section of the standard GRE does matter in econ, but typically only as a way to make the first cut (usually at 760 or 780, depending on the school). I don’t know much of the details here though.
6) very few people actually study for the GRE like I did—i.e. buy a prep book and work through it. This depresses their scores even though they’re much better quantitatively than I am.
Unsurprisingly since these are in when-I-though-of-them order, 1)-3) appeal to me the most, but 5) and 6) also seem plausible. I don’t see why 4) would bias the scores down instead of up so it seems unlikely a priori.
I was actually too lazy to study for my GRE, so I think I got like in the 600s on the math section (it had been a long time since I had studied any of that stuff); I realized while taking it that this was a stupid mistake and I was perfectly capable of answering everything, but the GRE cost too much for me to want to take it a second time. Oh well.
My guess is that there are a lot of grad schools (consider law schools, the standard advice is to not bother unless you can make the top 10, yet there are scores if not hundreds of active law schools), and few actually intend to do a PhD.
Did some research. The claim that the proposals are poorly written leaps out at me as immediately true. Here’s a website with successful grant applications, to be used as models to write them:
http://www.k12grants.org/samples/samples_index.htm
This is the first grant I pulled up (it’s not the first, but it -was- the first I felt competent to evaluate, concerning primarily technology):
http://www.k12grants.org/samples/grantkay.pdf
First, the horrible spelling, grammar, and punctuation leap out at me immediately. Second, the claim in the post that grant proposals are written to describe what they’re doing, rather than what they’re intending to achieve, holds up, for this grant at least.
http://www.k12grants.org/samples/MH%20grant.pdf
This proposal is the best-written I encountered. It describes the specific problems it intends to resolve and the specific solutions it intends to use. Unfortunately, the only evidence it introduces is the evidence that there is a problem. It doesn’t provide any evidence that its solutions work. Its stated “Method of Evaluation”, moreover, exactly mirrors the claims made in this post—it evaluates whether or not its solutions are implemented, NOT whether or not the problem is solved. (Goal #5 seems like an exception, but remember the stated problem is mental health issues.
http://www.k12grants.org/samples/TARGET.pdf
This proposal is the best I’ve encountered. It is horribly written, however. (You can skip past the pages and pages of documentation about how exactly the money will be spent to read the goals.) The accountability section has this (this is a proposal, essentially, to buy more modern computers for students and teachers, and to hire support staff):
Note that the accountability, as it pertains to this grant, is—wait for it—to make sure the grant money is spent as expected. You can change curriculum and instruction -without- the computers, remember. (I think this is a pretty sensible grant request, but the accountability measures it proposes provide no actual accountability. “Did we say what we were going to do? Yes? Then our grant was a success!”)
http://www.k12grants.org/samples/FLAP%20Narrative.pdf
Setting aside the fact that I’ve seen better writing from middle school students, this is actually a decently written grant. It has specific goals, implementations, and even has accountability. (Although it does seem confused about who or what is accountable to who or what; the accountability section reads rather like the author’s understanding of accountability meant the ability of students to measure their own improvements in performance. Notably omitted is a suggestion that the program’s success/improvement rate be compared to non-program success/improvement rates.)
http://www.k12grants.org/samples/2003_Library&Literacy.pdf
A well-written grant I can’t find fault with. (Except maybe its questionable notion of scientific evidence.)
So—some of the grants here definitely show symptoms of the problems indicated in the post. Some don’t. A couple of these had no business being granted. ALL of these grants were successful applications—that is, the grants were granted.
After this exercise, my position shifted from “This post is credible” to “This post exaggerates the extent of the problem to some degree, but remains a valid criticism of the grant system as it exists.”
And I tried to find a grant similar to the iPod/Makeover grant, and found this:
http://www.msmagiera.com/ipad-grant
Okay, not exactly analogous, as it at least pertains to education. However, given the grant’s self-evaluation criteria, student scores could plummet and the project could still call itself an overall success. (Actual improvement in student abilities only accounts for a fourth of their apparently unweighted criteria.)
I just read that grant in its entirety. I noticed one possible typo, but did not find other bad grammar or spelling.
The are asking for a grant to get equipment, primarily computers and software, for use in teaching students. It is not really a research project. What is the outcome hoped for from a grant like that? That students will be taught using these computers. They make a feint at claiming it will raise grades or enrolement, but really if I were a science teacher, my real goal would be to get the stuff and sit students down in front of it and teach them with it. I think that is pretty accurately reflected.
I’ll look at the ipad grant, and kudos for finding the site and bringing me that much closer to real contact with the kinds of grants under discussion.
Me too. Good thing they’re not trying to improve writing ability!
The VERY FIRST SENTENCE has minor punctuation issues and refers to “Excellence in Leaning (sp) Through Technology”—I refuse to believe that the original Senate bill being referred to failed to spell the word “Learning” correctly in its title. :-)
The second sentence puts a space before the colon for no apparent reason.
“The moneys this school is requesting” ⇒ should probably be “money”, though I’d accept argument to the contrary. “With request to …” ⇒ should probably be “With RESPECT to”
“This shows community support for improvement and a move forward with the support of a technology plan.” ⇒ You can tell what the writer is trying to say, but the writer is not actually saying it; the sentence is just broken.
“Teachers will...learn ho to integrate this technology” ⇒ should be “learn HOW to integrate...”
That’s just the first page, and it’s not even ALL the issues on the first page. Fortunately, the following pages are much better than the abstract page (which was painful). The second page is missing a bunch of hyphens—that’s a problem throughout—but otherwise not too bad.
Third page: “A desired outcome of this project is an increase in tile number of students taking high level science.” ⇒ change “tile number” to “total number” and possibly change “high level” to “high-level”
“By using MBL’s, less time is required” ⇒ change “MBL’s” to “MBLs”—it’s not a possessive.
“The purchase of this equipment would be in support of Colorado economy.” has a missing article; change it to ⇒ “would support THE Colorado economy”
“accommodate this set Up.” ⇒ “setup”.
Under IMPACT: “By obtaining these funds and implementing this program more students will be able to participate in hands on leaning” ⇒ again, it should be LEARNING, not LEANING. Also it’s “hands-on”, not “hands on”
“This science lab will be in place alter the grant period is over.” ⇒ AFTER the grant period, not ALTER.
Much of this suggests a very bad writer—less than 8th-grade level—who is using a spell-checker. But there some other mistakes that seem like the document might have been electronically scanned. For instance, the budget mentions “guides for teachers arid students” ⇒ should obviously be “teachers AND students” but I can’t imagine a human writer accidentally writing “arid” for “and” and “ri” does look an awful lot like “n”.
Agree with everything but:
If you look in an old enough style guide (the current standard is as you say), it will say to use an apostrophe when you pluralize an acronym. Wikipedia agrees.
Quotes from the .pdf, with my corrections:
respect, regard, or reference.
how
hardware, software
in–depth
principles
day–to–day
after
$2,250
Consistently ‘moneys’ is used where ‘money’ or ‘monies’ seems correct to me; I did not count this as an error despite not following a strict style guide. Most other ‘errors’ are very reasonably scanning errors rather than writing errors; the only error that couldn’t plausibly be a scanner error would be ‘principal’ for ‘principle’.
Overall, the writing was simplistic, sentences were short and simple, and would pass a technical writing test. Presented as a model for what complexity and intelligence level of grants are approved, that is very informative. Grant proposals (apparently) should be simple, repetitive, and full of Capitalized Buzzwords that are Important to the Right People.
I actually counted the really short sentences heavily against them mentally, probably too much. Owing to the way I parse sentences, reading the grant was like listening to William Shatner at his… not quite hammiest, but pretty close.
As far as the 2.250 thing, that’s actually not that uncommon outside English-speaking nations; see http://en.wikipedia.org/wiki/Decimal_point#Countries_using_Arabic_numerals_with_decimal_comma which lists countries which use decimals as thousands separators and commas as decimal marks. (That may actually help to explain the short sentences, come to think of it.)
An alternative explanation is offered here: http://lesswrong.com/lw/h5z/problems_in_education/8qqj (Specifically, that the document may have been electronically scanned; this could also account for other apparent spelling mistakes. Handwriting recognition is getting better, but is still far from perfect.)
How, exactly is it, that these people get hired in the first place?
I’ll put it this way: in the average GRE scores by intended field, education ranks below philosophy & STEM in every subtest, and various forms of education rank very low (early childhood education is, out of 50 groups, second from the bottom in 2 subtests and fifth from the bottom in the last subtest).
Average GRE is useless. Elementary teachers have far lower GRE scores than secondary school teachers, and are about average in verbal and below average in math. Secondary school content teachers are above average in verbal and average in math. However, close to half of all secondary school teachers get higher than 600 on the math section, which is more than the number of math and science teachers. While I suppose it’s possible that math and science teachers have terrible math scores and the English/history teachers are scoring those 600+ scores, I’m figuring it’s far more likely that math and science high school teachers have eminently respectable GRE scores in math, and that English/history teachers have higher than average verbal.
Anyone who claims that teachers are stupid is using propaganda instead of ETS data.
Cite for SAT scores and for GRE scores
What makes “higher than 600 on ONE section” a cutoff above which counts as an “eminently respectable” score?
Would you accept “mediocre”? ;-)
Education even ranks below Religion in every category. Also, Economics is only quantitatively better than Religion.
Not surprising, given my experience. Most religion majors I’ve met were relatively smart and often made fun of the more fundamentalist/evangelical types who typically were turned off by their religion classes. Religion majors seemed like philosophy-lite majors (which is consistent with the rankings).
Edit: Also, relative to Religion, econ has a bunch of poor english speakers that pull the other two categories down. (Note: the “analytical” section is/was actually a couple of very short essays)
Yes, given that economics is apparently one of the most lucrative fields around going by Caplan’s recent post on majors, it’s interesting that the econ students aren’t ranked even higher.
Chicken-and-egg problem: Non-economics majors don’t think economically enough to choose fields on the basis of their remuneration?
That seems to explain why Econ majors get a premium, but that doesn’t seem to explain why econ majors don’t rank higher, or am I missing something?
Those who can’t do...
Another category that jumped out at me—see all the public/education/business administration near the bottom. These are the people running institutions. The world is the way it is for a reason.
I notice that I’m confused: the maximum score on the Quantitative section is 800 (at that time), and Ph.D. econ programs won’t even consider you if you’re under a 780. The quantitative exam is actually really easy for math types. When you sign up for the GRE, you get a free CD with 2 practice exams. When I took it, I took the first practice exam without studying at all and got a 760 or so on the quantitiative section (within 10 pts). After studying I got a 800 on the second practice exam and on the actual exam, I got a 790. The questions were basic algebra for the most part with a bit of calculus and basic stat at the top end and a tricky question here and there. The exam was easy—really easy. I was a math major at a tiny / terrible liberal arts school; nothing like MIT or any self respecting state school. So it seems like it should be easy for anyone with a halfway decent mathematics background.
Now you’re telling me people intending to major in econ in grad school average a 706, and people intending to major in math average a 733? That’s low. Really low relative to my expectations. I would have expected a 730 in econ and maybe a 760 in math.
Possible explanations:
1) Tons of applicants who don’t want to believe that they aren’t cut out for their field create a long tail on the low side while the high side is capped at 800.
2) Master’s programs are, in general, more lenient and there are a large number of people who only intend to go to them, creating the same sort of long tail effect as above in 1).
3) There’s way more low-tier graduate programs than I thought in both fields willing to accept the average or even below average student.
4) Weirdness in how these fields are classified (e.g. I don’t see statistics there anywhere, is that included in math?)
5) the quantitative section of the standard GRE actually doesn’t matter if you’re headed to a math or physics program (someone in that field care to comment?). Note: the quantitative section of the standard GRE does matter in econ, but typically only as a way to make the first cut (usually at 760 or 780, depending on the school). I don’t know much of the details here though.
6) very few people actually study for the GRE like I did—i.e. buy a prep book and work through it. This depresses their scores even though they’re much better quantitatively than I am.
Unsurprisingly since these are in when-I-though-of-them order, 1)-3) appeal to me the most, but 5) and 6) also seem plausible. I don’t see why 4) would bias the scores down instead of up so it seems unlikely a priori.
I was actually too lazy to study for my GRE, so I think I got like in the 600s on the math section (it had been a long time since I had studied any of that stuff); I realized while taking it that this was a stupid mistake and I was perfectly capable of answering everything, but the GRE cost too much for me to want to take it a second time. Oh well.
Statistics does not seem to be broken out in the latest GRE scores I found: https://www.ets.org/s/gre/pdf/gre_guide.pdf I think statistics is almost always part of the math department.
My guess is that there are a lot of grad schools (consider law schools, the standard advice is to not bother unless you can make the top 10, yet there are scores if not hundreds of active law schools), and few actually intend to do a PhD.