Testing and certification is an extremely difficult thing to get right, and indeed I’m not sure anyone really has done this. It takes a lot of effort to write good tests. It is almost impossible to simultaneously reduce both false positives and false negatives. If the test is standardized or repeated, teachers teach to and students train for the test in preference to the subject matter. Ideally you want the test to be such that someone who prepares for the test is also preparing for the skills you wish to measure.
Are there any existing tests (e.g. bar exam, SAT, CCNA, etc.) that you would point to as suitable for this system? If not, what changes would need to be made to make them suitable?
I will admit that the current system of testing and certification is bad, and I remind myself that I need to watch out for status quo bias here.
Testing and certification is an extremely difficult thing to get right, and indeed I’m not sure anyone really has done this. It takes a lot of effort to write good tests. It is almost impossible to simultaneously reduce both false positives and false negatives. If the test is standardized or repeated, teachers teach to and students train for the test in preference to the subject matter. Ideally you want the test to be such that someone who prepares for the test is also preparing for the skills you wish to measure.
Are there any existing tests (e.g. bar exam, SAT, CCNA, etc.) that you would point to as suitable for this system? If not, what changes would need to be made to make them suitable?
I will admit that the current system of testing and certification is bad, and I remind myself that I need to watch out for status quo bias here.