Calibration Test with database of 150,000+ questions

Hi all,

I put this cal­ibra­tion test to­gether this morn­ing. It pulls from a trivia API of over 150,000 ques­tions so you should be able to take this many, many times be­fore you start see­ing re­peats.


A few notes:

1. The ques­tions are “Jeop­ardy” style ques­tions so the word­ing may be strange, and some of them might be im­pos­si­ble to an­swer with­out fur­ther con­text. On these just as­sign 0% con­fi­dence.

2. As the ques­tions are open-ended, there is no an­swer-check­ing mechanism. You have to be hon­est with your­self as to whether or not you got the right an­swer. Be­cause what would be the point of cheat­ing at a cal­ibra­tion test?

I can’t think of any­thing else. Please let me know if there are any fea­tures you would want to see added, or if there are any bugs, is­sues, etc.


As per sug­ges­tion I have moved this to the main sec­tion. Here are the changes I’ll be mak­ing soon:

  • La­bel the axes and in­clude an ex­pla­na­tion of cal­ibra­tion curves.

  • Make it so you can re­verse your last se­lec­tion in the event of a misclick.

Here are changes I’ll make even­tu­ally:

  • Create an ac­count sys­tem so you can store your re­sults on­line.

  • Move trivia DB over to my own server to al­low for flag­ging of bad/​unan­swer­able ques­tions.

Here are the changes that are done:

  • Change 0% to 0.1% and 99% to 99.9%

  • Added sec­ond graph which shows the fre­quency of your con­fi­dence se­lec­tions.

  • Color code the “right” and “wrong” but­tons and make them farther apart to pre­vent misclicks.

  • Store your re­sults lo­cally so that you can see your cal­ibra­tion over time.

  • Check to see if a ques­tion is blank and skip if so.