I have approximately zero knowledge about Zillow or the real estate business, so I can’t comment on whether this actually is what went wrong.
FWIW, like the tank or Microsoft Tay or the Amazon hiring system stories, Zillow seems quite dubious as an AI-risk/bias story despite its rapid canonization into the narrative.
The current narrative seems to trace mostly to the Zillow CEO’s initial analyst call where he blamed Zillow’s losses on the models, saving his face although neglecting to explain how his competitors in the same markets using similar models & datasets mysteriously did so much better, which then got repeated in the online echo chamber (ever more encrusted with opinions and speculation).
But almost immediately, anonymous comments on Reddit/HN & Zillow insiders talking to Bloomberg journalists told a different story: Zillow had two sets of models, and was well-aware that the secondary price model necessarily would lead to the winner’s curse & losses if they bought at the model’s high bid because they’d only win the houses they were overpaying for; they bought all the houses anyway because of a failure of management, which wanted to hit sales & growth targets and used the secondary rather than primary model as an excuse to do all the buys. This risk-hungry growth-buying strategy looked good in the short-term but then failed long-term in the predictable way, as has happened to many market-makers and companies in the past, when a predictably unpredictable market fluctuation happened.
Interesting, I added a note to the text highlighting this! I was not aware of that part of the story at all. That makes it more of a Moloch-example than a “mistaking adversarial for random”-example.
Yes, it is a cautionary lesson, just not the one people are taking it for; however, given the minimal documentation or transparency, there are a lot of better examples of short-term risk-heavy investments or blowing up (which are why Wall Street strictly segregates the risk-control departments from the trader and uses clawbacks etc) to tell, so the Zillow anecdote isn’t really worth telling in any context (except perhaps, like the other stories, as an example of low epistemic standards and how leprechauns are born).
FWIW, like the tank or Microsoft Tay or the Amazon hiring system stories, Zillow seems quite dubious as an AI-risk/bias story despite its rapid canonization into the narrative.
The current narrative seems to trace mostly to the Zillow CEO’s initial analyst call where he blamed Zillow’s losses on the models, saving his face although neglecting to explain how his competitors in the same markets using similar models & datasets mysteriously did so much better, which then got repeated in the online echo chamber (ever more encrusted with opinions and speculation).
But almost immediately, anonymous comments on Reddit/HN & Zillow insiders talking to Bloomberg journalists told a different story: Zillow had two sets of models, and was well-aware that the secondary price model necessarily would lead to the winner’s curse & losses if they bought at the model’s high bid because they’d only win the houses they were overpaying for; they bought all the houses anyway because of a failure of management, which wanted to hit sales & growth targets and used the secondary rather than primary model as an excuse to do all the buys. This risk-hungry growth-buying strategy looked good in the short-term but then failed long-term in the predictable way, as has happened to many market-makers and companies in the past, when a predictably unpredictable market fluctuation happened.
Interesting, I added a note to the text highlighting this! I was not aware of that part of the story at all. That makes it more of a Moloch-example than a “mistaking adversarial for random”-example.
Yes, it is a cautionary lesson, just not the one people are taking it for; however, given the minimal documentation or transparency, there are a lot of better examples of short-term risk-heavy investments or blowing up (which are why Wall Street strictly segregates the risk-control departments from the trader and uses clawbacks etc) to tell, so the Zillow anecdote isn’t really worth telling in any context (except perhaps, like the other stories, as an example of low epistemic standards and how leprechauns are born).