I don’t think that it’s reasonable to expect that secret criteria would stay secret once such a test would actually be used for anything. Sure, it could be kept a secret if there were a dozen people taking the test, of which the four who passed would get admitted to an exclusive club.
If there were ten thousand people taking the test, a thousand of which passed, I’d bet there’d be at least one who accidentally leaks it on the internet, from where it would immediately become public knowledge. (And at least a dozen who would willingly give up the answer if offered money for it, as would happen if there were anything at stake in this test.) It might work if such a test is obscure enough or not widely used, but not if it was used for anything that mattered to the test-takers and was open to many.
True, but I think that would be a problem with any test. I’m just trying to find a way around it since I think that as you add ways to avoid gaming, you both complicate and weaken the test. Perhaps a solution would be to test people without their knowledge, and reveal whether they succeeded or not at a later date.
I don’t think that it’s reasonable to expect that secret criteria would stay secret once such a test would actually be used for anything. Sure, it could be kept a secret if there were a dozen people taking the test, of which the four who passed would get admitted to an exclusive club.
If there were ten thousand people taking the test, a thousand of which passed, I’d bet there’d be at least one who accidentally leaks it on the internet, from where it would immediately become public knowledge. (And at least a dozen who would willingly give up the answer if offered money for it, as would happen if there were anything at stake in this test.) It might work if such a test is obscure enough or not widely used, but not if it was used for anything that mattered to the test-takers and was open to many.
True, but I think that would be a problem with any test. I’m just trying to find a way around it since I think that as you add ways to avoid gaming, you both complicate and weaken the test. Perhaps a solution would be to test people without their knowledge, and reveal whether they succeeded or not at a later date.