If anyone is thinking about creating their own, I would suggest questions with numerical answers so you can give upper and lower bounds of varying confidence, rather than trying to pick your confidence on a binary question and try to force binning or do some sort of filtering.
Also, this lets you give several probability estimates for each question.
I found How to Measure Anything pretty interesting in its thorough application of calibration and Fermi calculation to all sorts of problems, although I didn’t find the digressions into Excel very useful. Definitely recommended if you don’t already have the mental knack for Fermi stuff.
If anyone is thinking about creating their own, I would suggest questions with numerical answers so you can give upper and lower bounds of varying confidence, rather than trying to pick your confidence on a binary question and try to force binning or do some sort of filtering.
Also, this lets you give several probability estimates for each question.
Douglas Hubbard writes on the topic of calibration as well. He focuses on RW application of this stuff, and calibration is clearly a part of that.
His 1st book: http://www.amazon.com/How-Measure-Anything-Intangibles-ebook/dp/B001BPE8ZQ/ref=sr_1_3?ie=UTF8&s=books&qid=1258133710&sr=8-3
His site: http://www.hubbardresearch.com/dotnetnuke/
I found How to Measure Anything pretty interesting in its thorough application of calibration and Fermi calculation to all sorts of problems, although I didn’t find the digressions into Excel very useful. Definitely recommended if you don’t already have the mental knack for Fermi stuff.