Nice, I didn’t know OpenPhil had calibration training.
It is difficult to use SPIES for the calibration training—I kept running out of time when using my implementation in Python. To still compare the methods, I copied some questions and gave a confidence interval and SPIES estimate. Here are the results; I’ve only included 5 questions, but from what I’ve done, it seems SPIES helps me to narrow might 80% confidence intervals.
1. In which year was the US Open decided for the first time by ‘sudden death’?
You could get far more rapid feedback on the usefulness of this method by using it in a calibration training.
Nice, I didn’t know OpenPhil had calibration training.
It is difficult to use SPIES for the calibration training—I kept running out of time when using my implementation in Python. To still compare the methods, I copied some questions and gave a confidence interval and SPIES estimate. Here are the results; I’ve only included 5 questions, but from what I’ve done, it seems SPIES helps me to narrow might 80% confidence intervals.
1. In which year was the US Open decided for the first time by ‘sudden death’?
CI: 1900-2000
SPIES: 1938-2000 : 1900-1924 16.54%; 1925-1948 24.63%; 1949-1972 29.41%; 1973-1996 29.41%
Actual Value: 1990
2. In what year did Emerson Fittipaldi first win the World Championship?
CI: 1910-2010
SPIES: 1939-2010 : 1910-1935 18.18%; 1936-1960 11.36%; 1961-1985 36.36%; 1986-2010 34.09%
Actual Value: 1972
3. In what year was rayon first produced in the United States?
CI: 1780-2005
SPIES: 1836-1996 : 1780-1836 16.28%; 1837-1892 27.91%; 1893-1948 27.91%; 1949-2005 27.91%
Actual Value: 1910
4. When was the first Winter Olympics held?
CI: 1880-1980
SPIES: 1914-1980 : 1880-1905 13.04%; 1906-1930 21.74%; 1931-1955 26.09%; 1956-1980 39.13%
Actual Value: 1924
5. In which year did Frankie Goes to Hollywood form?
CI: 1910-2000
SPIES: 1938-2000 : 1910-1932 15.0%; 1933-1954 20.0%; 1955-1976 30.0%; 1977-2000 35.0%
Actual Value: 1980