gjm and Lumifer, thanks for the detailed discussion.
I want to clarify a few points, mainly regarding the context of what I’m writing. My goal was to give an intuition about multiplicity cropping up in different contexts in 400 words, not to explain the details of financial engineering or the difference between t and Z scores.
There are advanced approaches to annualizing Sharpe Ratios but the first method that people are taught is to:
Take the average daily return over a number of days, and multiply that by 252 (number of trading days in a year) to get the yearly return.
Take the daily standard deviation and multiply by sqrt(252) to get the yearly standard deviation.
Subtract the risk-free return from #1 and divide by #2.
Similarly, the basic way that people are taught to test for statistical significance is:
Calculate the average measurement.
Calculate the standard deviation of the average by dividing the sd of the sample by sqrt(sample size).
Subtract the null (usually 0) from #1 and divide by #2.
As a convention, Sharpe Ratios are standardized to a single year and statistics in science (e.g. a drug effectiveness) are standardized to a single average person. If everyone measured daily Sharpe Ratios (instead of yearly) and the effects of drugs on groups of 252 people at once, whether we multiply or divide would switch. But at the core, we’re doing the same thing: making a lot of assumptions about how something is distributed and then dividing the “excess” result for a standard unit by the SD of that unit.
I agree that in practice people look at those things very differently. In social psychology you get p 2) and go publish, while in stock-picking Sharpe Ratios rarely get anywhere close to 2, and you compare the ratios directly instead of thinking of them as p-values. Still, both measurements are equally affected by testing multiple hypotheses and reporting the best one. If someone tells me of a stock picking strategy (17th lag!) that has a Sharpe Ratio of 0.4 (as compared to the S&P’s 0.25) but they tried 20 strategies to get there, it’s worth as much as green jelly beans. That’s all I was trying to get at in the first half of the post, and I don’t think anyone disagrees on this point.
Heh, nope. Finance people (other than marketers) are very interested in empirical truth because for them the match between the map and the territory directly translates into money. Hope is not a virtue in finance.
And that’s exactly what I was trying to get at in the second half of the post.
My goal was to give an intuition about multiplicity
In which case you don’t need the digression into Sharpe ratios at all. It just distracts from the main point.
the first method that people are taught is to: 1. Take the average daily return over a number of days, and multiply that by 252
Err… If I may offer more advice, don’t breezily barge into subjects which are more complicated than they look.
The “average daily return” for people who are taught their first method usually means the arithmetic return (P1/P0 − 1). If so, you do NOT multiply that number by 252 because arithmetic returns are not additive across time. Log returns (log(P1/P0)) are, but people who are using log returns are usually already aware of how Sharpe ratios work.
the basic way that people are taught to test for statistical significance
This is testing the significance of the mean. I would probably argue that the most common context where people encounter statistical significance is a regression and the statistical significance in question is that of the regression coefficients. And for these, of course, it’s a bit more complicated.
Still, both measurements are equally affected by testing multiple hypotheses
I don’t understand what this means. If you do multiple tests and pick the best, any measurement is affected.
The “average daily return” for people who are taught their first method usually means the arithmetic return (P1/P0 − 1). If so, you do NOT multiply that number by 252 because arithmetic returns are not additive across time. Log returns (log(P1/P0)) are, but people who are using log returns are usually already aware of how Sharpe ratios work.
If your daily returns are so big that ln(P1/P0) is non-negligibly different from P1/P0 − 1, I’m interested in knowing what your investment strategy is. ;-)
In which case you don’t need the digression into Sharpe ratios at all. It just distracts from the main point.
I wrote a long post specifically about adjusting for multiplicity, this is just a follow up to demonstrate that multiplicity is a problem even if you don’t call what you measure “p-values”. I think you read that one, and commented that I shouldn’t use p-values at all. I agreed.
If you do multiple tests and pick the best, any measurement is affected.
You know this, I know this, but a lot of people from investment bankers to social psychologists don’t know that, or at least don’t understand it on a deep level. That’s probably also true of many of my blog readers.
I feel like we’re not really disagreeing. My point was “there are similarities between p-values and Sharpe Ratios, especially in the way they’re affected by multiplicity”. Your point was that they’re not exactly the same thing. OK.
If I may offer more advice, don’t breezily barge into subjects which are more complicated than they look.
Thanks, but I plan to follow the opposite advice. It’s a popular blog, not a textbook, and part of my goal in writing it is to learn stuff. For example, since I wrote the last post I learned a lot about Sharpe Ratios. I also think that the thrust of my post stands regardless of the exact parameters and assumptions for calculating Sharpe Ratios (which is a subject of textbooks).
but a lot of people from investment bankers to social psychologists don’t know that, or at least don’t understand it on a deep level
You’ll excuse me if I find myself a bit sceptical with respect to your opinion about what investment bankers understand on a deep level and what they don’t...
Your point was that they’re not exactly the same thing
Well, actually my point was that they are not the same thing at all and confusing them is a category error. Perhaps I didn’t express my point strongly enough :-P
It’s a popular blog, not a textbook, and part of my goal in writing it is to learn stuff.
Sure, it’s your blog. I just think it would be best not to mislead your readers.
gjm and Lumifer, thanks for the detailed discussion.
I want to clarify a few points, mainly regarding the context of what I’m writing. My goal was to give an intuition about multiplicity cropping up in different contexts in 400 words, not to explain the details of financial engineering or the difference between t and Z scores.
There are advanced approaches to annualizing Sharpe Ratios but the first method that people are taught is to:
Take the average daily return over a number of days, and multiply that by 252 (number of trading days in a year) to get the yearly return.
Take the daily standard deviation and multiply by sqrt(252) to get the yearly standard deviation.
Subtract the risk-free return from #1 and divide by #2.
Similarly, the basic way that people are taught to test for statistical significance is:
Calculate the average measurement.
Calculate the standard deviation of the average by dividing the sd of the sample by sqrt(sample size).
Subtract the null (usually 0) from #1 and divide by #2.
As a convention, Sharpe Ratios are standardized to a single year and statistics in science (e.g. a drug effectiveness) are standardized to a single average person. If everyone measured daily Sharpe Ratios (instead of yearly) and the effects of drugs on groups of 252 people at once, whether we multiply or divide would switch. But at the core, we’re doing the same thing: making a lot of assumptions about how something is distributed and then dividing the “excess” result for a standard unit by the SD of that unit.
I agree that in practice people look at those things very differently. In social psychology you get p 2) and go publish, while in stock-picking Sharpe Ratios rarely get anywhere close to 2, and you compare the ratios directly instead of thinking of them as p-values. Still, both measurements are equally affected by testing multiple hypotheses and reporting the best one. If someone tells me of a stock picking strategy (17th lag!) that has a Sharpe Ratio of 0.4 (as compared to the S&P’s 0.25) but they tried 20 strategies to get there, it’s worth as much as green jelly beans. That’s all I was trying to get at in the first half of the post, and I don’t think anyone disagrees on this point.
And that’s exactly what I was trying to get at in the second half of the post.
Glad to be of service :-)
In which case you don’t need the digression into Sharpe ratios at all. It just distracts from the main point.
Err… If I may offer more advice, don’t breezily barge into subjects which are more complicated than they look.
The “average daily return” for people who are taught their first method usually means the arithmetic return (P1/P0 − 1). If so, you do NOT multiply that number by 252 because arithmetic returns are not additive across time. Log returns (log(P1/P0)) are, but people who are using log returns are usually already aware of how Sharpe ratios work.
This is testing the significance of the mean. I would probably argue that the most common context where people encounter statistical significance is a regression and the statistical significance in question is that of the regression coefficients. And for these, of course, it’s a bit more complicated.
I don’t understand what this means. If you do multiple tests and pick the best, any measurement is affected.
If your daily returns are so big that ln(P1/P0) is non-negligibly different from P1/P0 − 1, I’m interested in knowing what your investment strategy is. ;-)
Once you upscale your daily returns by more than two orders of magnitude (that is, multiply them by 250), the difference becomes quite noticeable.
I wrote a long post specifically about adjusting for multiplicity, this is just a follow up to demonstrate that multiplicity is a problem even if you don’t call what you measure “p-values”. I think you read that one, and commented that I shouldn’t use p-values at all. I agreed.
You know this, I know this, but a lot of people from investment bankers to social psychologists don’t know that, or at least don’t understand it on a deep level. That’s probably also true of many of my blog readers.
I feel like we’re not really disagreeing. My point was “there are similarities between p-values and Sharpe Ratios, especially in the way they’re affected by multiplicity”. Your point was that they’re not exactly the same thing. OK.
Thanks, but I plan to follow the opposite advice. It’s a popular blog, not a textbook, and part of my goal in writing it is to learn stuff. For example, since I wrote the last post I learned a lot about Sharpe Ratios. I also think that the thrust of my post stands regardless of the exact parameters and assumptions for calculating Sharpe Ratios (which is a subject of textbooks).
You’ll excuse me if I find myself a bit sceptical with respect to your opinion about what investment bankers understand on a deep level and what they don’t...
Well, actually my point was that they are not the same thing at all and confusing them is a category error. Perhaps I didn’t express my point strongly enough :-P
Sure, it’s your blog. I just think it would be best not to mislead your readers.