I’m unclear on the benefits. I can see how it’s sometimes faster, but I’m unclear why faster is an important criterion, and it will sometimes be slower: if I quickly generated the first example in the post of 33,33,33,1, then I’d likely slow down a good deal trying to decide which of those 33s to turn into a 30 and which to turn into a 40.
Whereas it seems clearly valuable to encourage a habit of having the probabilities sum to 100%. That’s not for the scoring rule, it’s for developing good intuitions.
if I quickly generated the first example in the post of 33,33,33,1
If you’re doing a test with a bunch of these questions, and you have the same probabilities to choose from every time, then I’d expect you to quickly generate those numbers instead, and not have this problem.
I’m unclear on the benefits. I can see how it’s sometimes faster, but I’m unclear why faster is an important criterion, and it will sometimes be slower: if I quickly generated the first example in the post of 33,33,33,1, then I’d likely slow down a good deal trying to decide which of those 33s to turn into a 30 and which to turn into a 40.
Whereas it seems clearly valuable to encourage a habit of having the probabilities sum to 100%. That’s not for the scoring rule, it’s for developing good intuitions.
If you’re doing a test with a bunch of these questions, and you have the same probabilities to choose from every time, then I’d expect you to quickly generate those numbers instead, and not have this problem.