It’s hard to write good tests. It’s even harder to write good standardized tests that have to be administered to huge numbers of people, within a reasonable time, in purely written form, and graded consistently, again within reasonable time[1].
… and there are a ton of practical and political pressures that make it even harder.
People don’t actually agree on what they’re trying to teach, why it’s important to teach it, what constitutes good performance, what’s essential versus what’s “extra”, what it’s fair to test, what forms even of practically accessible testing are fair to use, etc. That list of critical competencies in the state curriculum wasn’t handed down by God. It was negotiated, and it’s at best a poor approximation of what anybody wants. And there was still more negotiation involved in making a test out of it.
Here’s an example of a practical and maybe political limitation: If you have a diagnosed learning disability, certified by somebody with the right piece of paper on their wall, you can (often) get extra time on tests. But you still have to take the tests, because the bottom line is that, disability or no, you still have to learn the material. You don’t get excused from actually getting the required understanding, even if it’s harder for you. You’re just given the opportunity to show that you can do it anyway.
The Good and the Great have agreed among themselves that, after all, what they care about is your understanding of, say, this or that kind of math, which they believe is best demonstrated by your ability to solve whatever problems are on the test. But they don’t really care about speed. It’s not a footrace, and nobody’s training (or hiring) people for an assembly line. If anybody in 2025 cared about actually solving the problems fast, or indeed accurately, they’d use a machine to do it anyway[2].
So we’ve established that speed doesn’t matter, at least not enough to withhold grades or credentials over it. It’s not a critical part of the “X” you’re trying to test.
OK, then why was the test timed in the first place? You don’t care about speed, remember? So why doesn’t everybody taking the test get all the time they feel they need? Your test is measuring the wrong thing!
The main real answer is probably that it’d be costly and administratively inconvenient to reserve a big room for, say, twice as long, and costly and personally inconvenient for somebody to sit there and proctor for that long. The second real answer is probably that timed tests are traditional, dammit.
So if it’s even possible to do what you’re suggesting, it’s going to be a lot of work. It’s going to have a cost. You’ll have to give something up to do it. Before you do that, you should know what value you’re getting in exchange.
So what value is this fancy test actually supposed to produce, once you have it?
The “bonus questions” you suggest probably make all time issues worse. Part of test design is to devote resources, including student time and grader time, to questions that are likely to get missed often enough to give you useful information about many of your testees.
Math problems machines can’t solve these days aren’t going to be good for tests, because most testees won’t be able to solve them either, at least not within the available time. We keep coming back to time.
OK, then why was the test timed in the first place? You don’t care about speed, remember?
Someone who takes a very long time to get answers may be more likely to completely forget the information in the future, may be less able to use the information in real world situations, etc. even though he eventually managed to write the answer on the test. Someone who takes a long time to write the answer down because he has a disability may not have these problems.
So while we don’t care about speed by itself, we care about problems that are correlated with speed, and they may be less so for the disabled.
So what value is this fancy test actually supposed to produce, once you have it?
Comparison between schools, for starters. As an information for parents where to put their kids.
Today, there are many schools that inflate grades, because people use “good grades” as a proxy for “more knowledge”. Which would be closer to the truth if we had better tests, but today it is too easy to cheat.
In absence of objective testing, the competition between schools is mostly about marketing. This is how you get schools that spend a lot of money on sports and various shiny toys, but provide mediocre education.
I have taught in different schools, so I know from personal experience that there is a huge difference between how much education the kids get in different schools. But even if a parent trying to get the best education for their child asks me, it is almost impossible for me to prove any of this; there is nothing legible to point at, other than say “trust me, that school is utterly dysfunctional, but they give every student good grades, and they have an army of lawyers to threaten any teacher who talks about what happens there, and the director is a charming psychopath who can convince most parents in a 1:1 talk that this is the best school ever, so the parents are happy to give him lots of money believing that they sacrifice for the good of their children”. (Yes, I have a specific school in mind. Not in USA.)
It’s hard to write good tests. It’s even harder to write good standardized tests that have to be administered to huge numbers of people, within a reasonable time, in purely written form, and graded consistently, again within reasonable time[1].
… and there are a ton of practical and political pressures that make it even harder.
People don’t actually agree on what they’re trying to teach, why it’s important to teach it, what constitutes good performance, what’s essential versus what’s “extra”, what it’s fair to test, what forms even of practically accessible testing are fair to use, etc. That list of critical competencies in the state curriculum wasn’t handed down by God. It was negotiated, and it’s at best a poor approximation of what anybody wants. And there was still more negotiation involved in making a test out of it.
Here’s an example of a practical and maybe political limitation: If you have a diagnosed learning disability, certified by somebody with the right piece of paper on their wall, you can (often) get extra time on tests. But you still have to take the tests, because the bottom line is that, disability or no, you still have to learn the material. You don’t get excused from actually getting the required understanding, even if it’s harder for you. You’re just given the opportunity to show that you can do it anyway.
The Good and the Great have agreed among themselves that, after all, what they care about is your understanding of, say, this or that kind of math, which they believe is best demonstrated by your ability to solve whatever problems are on the test. But they don’t really care about speed. It’s not a footrace, and nobody’s training (or hiring) people for an assembly line. If anybody in 2025 cared about actually solving the problems fast, or indeed accurately, they’d use a machine to do it anyway[2].
So we’ve established that speed doesn’t matter, at least not enough to withhold grades or credentials over it. It’s not a critical part of the “X” you’re trying to test.
OK, then why was the test timed in the first place? You don’t care about speed, remember? So why doesn’t everybody taking the test get all the time they feel they need? Your test is measuring the wrong thing!
The main real answer is probably that it’d be costly and administratively inconvenient to reserve a big room for, say, twice as long, and costly and personally inconvenient for somebody to sit there and proctor for that long. The second real answer is probably that timed tests are traditional, dammit.
So if it’s even possible to do what you’re suggesting, it’s going to be a lot of work. It’s going to have a cost. You’ll have to give something up to do it. Before you do that, you should know what value you’re getting in exchange.
So what value is this fancy test actually supposed to produce, once you have it?
The “bonus questions” you suggest probably make all time issues worse. Part of test design is to devote resources, including student time and grader time, to questions that are likely to get missed often enough to give you useful information about many of your testees.
Math problems machines can’t solve these days aren’t going to be good for tests, because most testees won’t be able to solve them either, at least not within the available time. We keep coming back to time.
Someone who takes a very long time to get answers may be more likely to completely forget the information in the future, may be less able to use the information in real world situations, etc. even though he eventually managed to write the answer on the test. Someone who takes a long time to write the answer down because he has a disability may not have these problems.
So while we don’t care about speed by itself, we care about problems that are correlated with speed, and they may be less so for the disabled.
Comparison between schools, for starters. As an information for parents where to put their kids.
Today, there are many schools that inflate grades, because people use “good grades” as a proxy for “more knowledge”. Which would be closer to the truth if we had better tests, but today it is too easy to cheat.
In absence of objective testing, the competition between schools is mostly about marketing. This is how you get schools that spend a lot of money on sports and various shiny toys, but provide mediocre education.
I have taught in different schools, so I know from personal experience that there is a huge difference between how much education the kids get in different schools. But even if a parent trying to get the best education for their child asks me, it is almost impossible for me to prove any of this; there is nothing legible to point at, other than say “trust me, that school is utterly dysfunctional, but they give every student good grades, and they have an army of lawyers to threaten any teacher who talks about what happens there, and the director is a charming psychopath who can convince most parents in a 1:1 talk that this is the best school ever, so the parents are happy to give him lots of money believing that they sacrifice for the good of their children”. (Yes, I have a specific school in mind. Not in USA.)