In a twitter thread, Eliezer Yudkowsky challenges the belief in EMH due to the seemingly illogical behavior of the markets with respect to the COVID-19. In the replies, Robin Hanson asks the obvious question: “what exactly is your better theory?”
I’ve been thinking a lot about EMH in recent years and would like to propose a different formulation of EMH—a one that I believe resolves some of the obvious issues with the current theory while preserving many of its conclusions and underlying logic at large. I will also suggest in which situations profit can be made and why the market was indeed inefficient in the COVID-19 situation.
Consider the following question:
If you feed a powerful AGI with almost unlimited computation resources all the public data we have regarding stocks and businesses—will it be able to beat the market?
If you believe in EMH your answer should be a “no.” But this answer seems wrong.
It’s hard to believe that a god-like AGI wouldn’t be able to find any competitive advantage in analyzing the data given practically unlimited resources. The underlying cognitive process of beating the market is the same as any other sufficiently hard cognitive task, a search in highly complex multidimensional space.
Having large amounts of computing power should help you to achieve better results, even without private information.
But yet we have those empirical results that show that traders having a very hard time to beat the market using public data. So I would like to suggest a different version of EMH that solves both of these issues. And the rule can be phrased in the following way:
“In the long run, the costs of analyzing public data in order to obtain alpha will be equal to the gained alpha.”
Intuition:
Think of crypto mining as an equivalent, if you are able to mine crypto for less money than what it’s worth you can make a profit (equivalent to alpha). But your competitors will do the same – raising the difficulty of mining, so in the long run in a competitive market, you can expect the mining expenses will be very close to the value of the mined crypto.
In a similar way, companies that invest in the market spend money on analysts and experts to find inefficiencies using public data and as long this is profitable the market should hire analysts to analyze data until the cost of analyzing data is equal to the alpha gained. So in an efficient enough market, you can expect that any marginal investment in data analysis will prove to be unprofitable.
More considerations
This model is obviously somewhat of a spherical cow, the real world is much more complex and it’s worthwhile to discuss some of the special cases and complexities in the light of this model.
The first thing worth noting is that compared to the crypto mining algorithm which is usually standard across miners – meaning everyone needs to solve the same hash function. In the case of stock markets, the multidimensional search is more complex, domain-level knowledge and deep understanding of industries could be very helpful to get a competitive advantage while these factors don’t exist in the crypto case. The practical implication means that someone might have a competitive advantage due to unrelated expertise which will grant him an ability to beat the market in a more cost-efficient way than its competitors.
Another important issue is the phenomenon of reflexivity, meaning the beliefs of the investors change the reality itself and by this creating very complex loops in the market. An interesting example could be the belief in EMH itself, as more people believe that it’s impossible to beat the markets the Ratio of dumb money (index funds) will become higher and will thus lower the data analysis costs you need to get alpha.
The last thing worth noting is that the rule is correct in the long run and on an average and stable situation, meaning when there are radical changes in the costs of data analysis due to rapid market changes it’s much easier to gain alpha. Returning to the crypto example, it would be equivalent to someone changing suddenly the hash functions to be better processed by GPUs instead of CPUs. So in the transition period, we can definitely expect players that have a competitive advantage in the ability to transition quickly to GPUs to make a profit.
COVID-19 Crash and practical considerations
In light of these points consider the COVID-19 market crash: the ability to predict the market suddenly shifted from the day to day necessary data analysis for stock price prediction that revolves more around business KPIs and geopolitical processes to understanding pandemics. How many of the Wall Street analysts are pandemic experts would you think? Probably very few. The rules have changes and prior data analysis resources (currently hired analysts) became suddenly very inefficient.
What players have a competitive advantage when the rules change?
The first group is domain experts; a hobbyist investor pandemics expert can probably outcompete the market on average in this specific scenario.
I’m not a pandemics expert but as part of my undergraduate engineering degree, I worked for a year on a simulation model of a spread of pandemic flu, after reading about the COVID-19 I’ve sold all my stocks at the end of January because I believed I had a domain knowledge advantage over the market.
The second group is nimble generalists that can adjust and learn quickly while big players are much slower to move. Wei Dai who made 700% profit on his bet probably belongs to this category. The classic metaphor in these cases is thinking about turning quickly a small fishing boat vs a cruiser. In a normal scenario, we can expect the large players will outcompete the small ones due to size advantages, but when dealing with black swans that require quickness small players might have the advantage.
Refactoring EMH – Thoughts following the latest market crash
In a twitter thread, Eliezer Yudkowsky challenges the belief in EMH due to the seemingly illogical behavior of the markets with respect to the COVID-19. In the replies, Robin Hanson asks the obvious question: “what exactly is your better theory?”
I’ve been thinking a lot about EMH in recent years and would like to propose a different formulation of EMH—a one that I believe resolves some of the obvious issues with the current theory while preserving many of its conclusions and underlying logic at large. I will also suggest in which situations profit can be made and why the market was indeed inefficient in the COVID-19 situation.
Consider the following question:
If you feed a powerful AGI with almost unlimited computation resources all the public data we have regarding stocks and businesses—will it be able to beat the market?
If you believe in EMH your answer should be a “no.” But this answer seems wrong.
It’s hard to believe that a god-like AGI wouldn’t be able to find any competitive advantage in analyzing the data given practically unlimited resources. The underlying cognitive process of beating the market is the same as any other sufficiently hard cognitive task, a search in highly complex multidimensional space.
Having large amounts of computing power should help you to achieve better results, even without private information.
But yet we have those empirical results that show that traders having a very hard time to beat the market using public data. So I would like to suggest a different version of EMH that solves both of these issues. And the rule can be phrased in the following way:
“In the long run, the costs of analyzing public data in order to obtain alpha will be equal to the gained alpha.”
Intuition:
Think of crypto mining as an equivalent, if you are able to mine crypto for less money than what it’s worth you can make a profit (equivalent to alpha). But your competitors will do the same – raising the difficulty of mining, so in the long run in a competitive market, you can expect the mining expenses will be very close to the value of the mined crypto.
In a similar way, companies that invest in the market spend money on analysts and experts to find inefficiencies using public data and as long this is profitable the market should hire analysts to analyze data until the cost of analyzing data is equal to the alpha gained. So in an efficient enough market, you can expect that any marginal investment in data analysis will prove to be unprofitable.
More considerations
This model is obviously somewhat of a spherical cow, the real world is much more complex and it’s worthwhile to discuss some of the special cases and complexities in the light of this model.
The first thing worth noting is that compared to the crypto mining algorithm which is usually standard across miners – meaning everyone needs to solve the same hash function. In the case of stock markets, the multidimensional search is more complex, domain-level knowledge and deep understanding of industries could be very helpful to get a competitive advantage while these factors don’t exist in the crypto case. The practical implication means that someone might have a competitive advantage due to unrelated expertise which will grant him an ability to beat the market in a more cost-efficient way than its competitors.
Another important issue is the phenomenon of reflexivity, meaning the beliefs of the investors change the reality itself and by this creating very complex loops in the market. An interesting example could be the belief in EMH itself, as more people believe that it’s impossible to beat the markets the Ratio of dumb money (index funds) will become higher and will thus lower the data analysis costs you need to get alpha.
The last thing worth noting is that the rule is correct in the long run and on an average and stable situation, meaning when there are radical changes in the costs of data analysis due to rapid market changes it’s much easier to gain alpha. Returning to the crypto example, it would be equivalent to someone changing suddenly the hash functions to be better processed by GPUs instead of CPUs. So in the transition period, we can definitely expect players that have a competitive advantage in the ability to transition quickly to GPUs to make a profit.
COVID-19 Crash and practical considerations
In light of these points consider the COVID-19 market crash: the ability to predict the market suddenly shifted from the day to day necessary data analysis for stock price prediction that revolves more around business KPIs and geopolitical processes to understanding pandemics. How many of the Wall Street analysts are pandemic experts would you think? Probably very few. The rules have changes and prior data analysis resources (currently hired analysts) became suddenly very inefficient.
What players have a competitive advantage when the rules change?
The first group is domain experts; a hobbyist investor pandemics expert can probably outcompete the market on average in this specific scenario.
I’m not a pandemics expert but as part of my undergraduate engineering degree, I worked for a year on a simulation model of a spread of pandemic flu, after reading about the COVID-19 I’ve sold all my stocks at the end of January because I believed I had a domain knowledge advantage over the market.
The second group is nimble generalists that can adjust and learn quickly while big players are much slower to move. Wei Dai who made 700% profit on his bet probably belongs to this category. The classic metaphor in these cases is thinking about turning quickly a small fishing boat vs a cruiser. In a normal scenario, we can expect the large players will outcompete the small ones due to size advantages, but when dealing with black swans that require quickness small players might have the advantage.