This is an interesting idea. However, the post assumes that 1) there is (or should be) one correct answer, 2) which is of the form: (1, 0, 0, 0) or a permutation thereof, and 3) the material is independent of the system (does not include probability, for example). Implementing this reasonably might require figuring out how to apply the system if that isn’t the case, and how to integrate with the underlying material in the event that the material isn’t independent of probability.
1. will probably be an issue as a result of errors. (Like a multiple choice question where “the correct answer” is “dirt” and that’s 2 of the 4 choices.)
2 and 3 are kind of the same.
Errata:
In other words, for each possibility i, the student loses the square of the distance between his answer qi and the true answer (0% or 100%).
the student loses the sum of the squares of the difference
Or just “the difference” (Euclidean norm).
Credence p4=[1%] in Lugano, but answers q4=0%.
The sum of the probabilities is 1.
this naive scoring incentivizes the exaggeration of beliefs towards deterministic answer.
Choosing the maximum probability, or randomizing if indifferent?
However, the post assumes that 1) there is (or should be) one correct answer, 2) which is of the form: (1, 0, 0, 0) or a permutation thereof, and 3) the material is independent of the system (does not include probability, for example).
These are assumed for the sake of explanation, but none are necessary; in fact, the scoring rule and analysis go through verbatim if you have questions with multiple answers in the form of arbitrary vectors of numbers, even if they have randomness. The correct choice is still to guess, for each potential answer, your expectation of that answer’s realized result.
This is an interesting idea. However, the post assumes that 1) there is (or should be) one correct answer, 2) which is of the form: (1, 0, 0, 0) or a permutation thereof, and 3) the material is independent of the system (does not include probability, for example). Implementing this reasonably might require figuring out how to apply the system if that isn’t the case, and how to integrate with the underlying material in the event that the material isn’t independent of probability.
1. will probably be an issue as a result of errors. (Like a multiple choice question where “the correct answer” is “dirt” and that’s 2 of the 4 choices.)
2 and 3 are kind of the same.
Errata:
the student loses the sum of the squares of the difference
Or just “the difference” (Euclidean norm).
The sum of the probabilities is 1.
Choosing the maximum probability, or randomizing if indifferent?
These are assumed for the sake of explanation, but none are necessary; in fact, the scoring rule and analysis go through verbatim if you have questions with multiple answers in the form of arbitrary vectors of numbers, even if they have randomness. The correct choice is still to guess, for each potential answer, your expectation of that answer’s realized result.