Maybe someone else commented something similar here, but I encountered this issue for the first time in an internship in a bank in 2007 where we had a fake investment competition. An option trader told me that a competition is really a binary option, with 0 value if less than strike price and a big reward if above. This means you want to maximize the value of the option which in this case means maximizing volatility. I crushed the portfolio competition by betting on leveraged financial instruments correlated to oil price (the biggest volatility investment at the time).
Another issue I have not explored with GJopen is the possibility of adjusting for past over or under estimates. For example, if you for some period in the beginning forecasted 70% probability of A and 30% of B and then mid way through the question get new data indicating that B is actually 70% likely, you might not want to put 70% but some higher percentage to compensate for the period before having a too low probability. So you can artificially lower your Brier score due to the forecasts being made over time. I am sure there are ways to account for that, I just have not had time to think about it.
an internship in a bank in 2007...I crushed the portfolio competition by betting on leveraged financial instruments correlated to oil price (the biggest volatility investment at the time).
This is such a common failure mode of stock competitions. Like every time you read about one of those old highschool stock market contests, back when newspapers printed stock listings, the story is invariably some bright kid realizing that trying to be Warren Buffett is for suckers because you get nothing if you don’t come in first, and YOLOing it on the most volatile or meme stock they can find. Embarrassing to see a bank with actual traders make that mistake - ‘heads I win tails you lose’ is the last lesson you want to be teaching your prospective traders… (Now, I’m not saying that your bank caused the 2008 crash and worldwide recession, but you’ll admit that it’s suspicious that it happened right before!)
Apologies, I don’t often come to this forum and only now saw your response. Fair point on teaching the wrong lesson. But to BofAs defence they had a really good workshop for us interns on betting under uncertainty. We had to answer various questions like how long is the Nile. The answer has to be a range and we could set it ourselves. We could put 0 − 10^99 on every answer and get all answers correct. Nobody got more than half right (mostly top grade ivy League grads) or so which really made me humble to overconfidence. They really did try to hammer in proper risk adjustment. That said, how traders bonuses were tied to their profit and loss was not ideal i think—i am pretty sure there were no negative bonuses. The bonuses were like the portfolio competition!
For example, if you for some period in the beginning forecasted 70% probability of A and 30% of B and then mid way through the question get new data indicating that B is actually 70% likely, you might not want to put 70% but some higher percentage to compensate for the period before having a too low probability
As it happens, I don’t think that’s the case. Like, ignoring the issues in the post/paper, the Brier score is proper, and this is regardless of how many points you’ve lost in the past.
Yeah I’m not too confident about it, i did not spend time looking into it. But I think it is >30% likely you can compensate for past over or under estimations.
Maybe someone else commented something similar here, but I encountered this issue for the first time in an internship in a bank in 2007 where we had a fake investment competition. An option trader told me that a competition is really a binary option, with 0 value if less than strike price and a big reward if above. This means you want to maximize the value of the option which in this case means maximizing volatility. I crushed the portfolio competition by betting on leveraged financial instruments correlated to oil price (the biggest volatility investment at the time).
Another issue I have not explored with GJopen is the possibility of adjusting for past over or under estimates. For example, if you for some period in the beginning forecasted 70% probability of A and 30% of B and then mid way through the question get new data indicating that B is actually 70% likely, you might not want to put 70% but some higher percentage to compensate for the period before having a too low probability. So you can artificially lower your Brier score due to the forecasts being made over time. I am sure there are ways to account for that, I just have not had time to think about it.
This is such a common failure mode of stock competitions. Like every time you read about one of those old highschool stock market contests, back when newspapers printed stock listings, the story is invariably some bright kid realizing that trying to be Warren Buffett is for suckers because you get nothing if you don’t come in first, and YOLOing it on the most volatile or meme stock they can find. Embarrassing to see a bank with actual traders make that mistake - ‘heads I win tails you lose’ is the last lesson you want to be teaching your prospective traders… (Now, I’m not saying that your bank caused the 2008 crash and worldwide recession, but you’ll admit that it’s suspicious that it happened right before!)
Apologies, I don’t often come to this forum and only now saw your response. Fair point on teaching the wrong lesson. But to BofAs defence they had a really good workshop for us interns on betting under uncertainty. We had to answer various questions like how long is the Nile. The answer has to be a range and we could set it ourselves. We could put 0 − 10^99 on every answer and get all answers correct. Nobody got more than half right (mostly top grade ivy League grads) or so which really made me humble to overconfidence. They really did try to hammer in proper risk adjustment. That said, how traders bonuses were tied to their profit and loss was not ideal i think—i am pretty sure there were no negative bonuses. The bonuses were like the portfolio competition!
As it happens, I don’t think that’s the case. Like, ignoring the issues in the post/paper, the Brier score is proper, and this is regardless of how many points you’ve lost in the past.
Yeah I’m not too confident about it, i did not spend time looking into it. But I think it is >30% likely you can compensate for past over or under estimations.
I’d bet against that at 1:5, i.e., against the proposition that the optimal forecast is not subject to your previous history