Yeah, and it doesn’t adjust for taxes there either. I thought this was less of an issue when comparing rents to owning though, as the same error should affect both equally.
SimonM
Karma: 886
This doesn’t seem to account for property taxes, which I expect would change the story quite a bit for the US.
This seems needlessly narrow minded. Just because AI is better than humans doesn’t make it uniformly better than humans in all subtasks of chess.
I don’t know enough about the specifics that this guy is talking about (I am not an expert) but I do know that until the release of NN-based algorithms most top players were still comfortable talking about positions where the computer was mis-evaluating positions soon out of the opening.
To take another more concrete example—computers were much better than humans in 2004, and yet Peter Leko still managed to refute a computer prepared line OTB in a world championship game.
Agreed—as I said, the most important things are compute and dilligence. Just because a large fraction of the top games are draws doesn’t really say much about whether or not there is an edge being given by the humans (A large fraction of elite chess games are draws, but no-one doubts there are differences in skill level there). Really you’d want to see Jon Edward’s setup vs a completely untweaked engine being administered by a novice.
I believe the answer is potentially. The main things which matter in high-level correspondence chess are:
Total amount of compute available to players
Not making errors
Although I don’t think either of those are really relevant. The really relevant bit is (apparently) planning:
For me, the key is planning, which computers do not do well — Petrosian-like evaluations of where pieces belong, what exchanges are needed, and what move orders are most precise within the long-term plan.
(From this interview with Jon Edwards (reigning correspondence world champion) from New In Chess)
I would highly recommend the interview on Perpetual Chess podcast also with Jon Edwards which I would also recommend.
I’ll leave you with this final quote, which has stuck with me for ages:
The most important game in the Final was my game against Osipov. I really hoped to win in order to extend my razor-thin lead, and the game’s 119 moves testify to my determination. In one middlegame sequence, to make progress, I had to find a way to force him to advance his b-pawn one square, all while avoiding the 50-move rule. I accomplished the feat in 38 moves, in a sequence that no computer would consider or find. Such is the joy of high-level correspondence chess. Sadly, I did not subsequently find a win. But happily, I won the Final without it!
How to Interpret Prediction Market Prices as Probabilities
I agree, as I said here
Just in case anyone is struggling to find the relevant bits of the the codebase, my best guess is the link for the collections folder in github is now here.
You are looking in “views.ts” eg …/collections/comments/views.ts
The best thing to search for (I found) was ”.addView(” and see what fits your requirements
I feel in all these contexts odds are better than log-odds.
Log-odds simplifies Bayesian calculations: so does odds. (The addition becomes multiplication)
Every number is meaningful: every positive number is meaningful and the numbers are clearer. I can tell you intuitively what 4:1 or 1:4 means. I can’t tell you what −2.4 means quickly, especially if I have to keep specifying a base.
Certainty is infinite: same is true for odds
Negation is the complement and 0 is neutral: Inverse is the complement and 1 is neutral. 1:1 means “I don’t know” and 1:x is the inverse of x:1. Both ot these are intuitive to me.
No—I think probability is the thing supposed to be a martingale, but I might be being dumb here.
Is Metaculus Slow to Update?
So, what do you think? Does this method seem at all promising? I’m debating with myself whether I should begin using SPIES on Metaculus or elsewhere.
I’m not super impressed tbh. I don’t see “give a 90% confidence interval for x” as a question which comes up frequently? (At least in the context of eliciting forecasts and estimates from humans—it comes up quite a bit in data analysis).
For example, I don’t really understand how you’d use it as a method on Metaculus. Metaculus has 2 question types—binary and continuous. For binary you have to give the probability an event happens—not sure how you’d use SPIES to help here. For continuous you are effectively doing the first step of SPIES—specifying the full distribution.
If I was to make a positive case for this, it would be—forcing people to give a full distribution results in better forecasts for sub-intervals. This seems an interesting (and plausible claim) but I don’t find anything beyond that insight especially valuable.
17. Unemployment below five percent in December: 73 (Kalshi said 92% that unemployment never goes above 6%; 49 from Manifold)
I’m not sure exactly how you’re converting 92% unemployment < 6% to < 5%, but I’m not entirely convinced by your methodology?15. The Fed ends up doing more than its currently forecast three interest rate hikes: None (couldn’t find any markets)
Looking at the SOFR Dec-22 3M futures 99.25/99.125 put spread on the 14-Feb, I put this probability at ~84%.
Thanks for doing this, I started doing it before I saw your competition and then decided against since it would have made cheating too easy. (Also why I didn’t enter)
And one way to accomplish that would be to bet on what percentage of bets are on “uncertainty” vs. a prediction.
How do you plan on incentivising people to bet on “uncertainty”? All the ways I can think of lead to people either gaming the index, or turning uncertainty into a KBC.
The market and most of the indicators you mentioned would be dominated by the 60 that placed large bets
I disagree with this. Volatility, liquidity, # predictors, spread of forecasts will all be affected by the fact that 20 people aren’t willing to get involved. I’m not sure what information you think is being lost by people stepping away? (I guess the difference between “the market is wrong” and “the market is uninteresting”?)
There are a bunch of different metrics which you could look at on a prediction market / prediction platform to gauge how “uncertain” the forecast is:
Volatility—if the forecast is moving around quite a bit, there are two reasons:
Lots of new information arriving and people updating efficiently
There is little conviction around “fair value” so traders can move the price with little capital
Liquidity—if the market is 49.9 / 50.1 in millions of dollars, then you can be fairly confident that 50% is the “right” price. If the market is 40 / 60 with $1 on the bid and $0.50 on the offer, I probably wouldn’t be confident the probability lies between 40 and 60, let along “50% is the right number”. (The equivalent on prediction platforms might be number of forecasters, although CharlesD has done some research on this which suggests there’s little addition value being added by large numbers of forecasters)
“Spread of forecasts”—on Metaculus (for example) you can see a distribution of people’s forecasts. If everyone is tightly clustered around 50% that (usually) gives me more confidence that 50% is the right number than if they are widely spread out
Prediction markets function best when liquidity is high, but they break completely if the liquidity exceeds the price of influencing the outcome. Prediction markets function only in situations where outcomes are expensive to influence.
There are a ton of fun examples of this failing:
Option expiries (I don’t have a good single link for this)
I don’t know enough about how equities trade during earnings, but I do know a little about how some other products trade during data releases and while people are speaking.
In general, the vast, vast, vast majority of liquidity is withdrawn from the market before the release. There will be a few stale orders people have left by accident + a few orders left in at levels deemed ridiculously unlikely. As soon as the data is released, the fastest players will general send quotes making a (fairly wide market) around their estimate of the fair price. Over time (and here we’re still talking very fast) more players will come in, firming up that new market.
The absolute level of money which is being made during this period is relatively small. It’s not like the first person to see the report gets to trade at the old price, they get to trade with any stale orders—the market just reprices with very little trading volume.
All of the money-making value was redeemed before people like you and me even had a chance to trade. Right?
Correct, you absolutely did not have the chance to be involved in this trade unless you work at one of a handful of firms which have spent 9 figure sums on doing this really, really well.
I agree identifying model failure is something people can be good at (although I find people often forget to consider it). Pricing it they are usually pretty bad at.
I assume this was supposed to be a WarGames reference, in which case I think it should be a “nice” game of chess.