It can tell you something about existing interventions in a variable. In the US, for instance, we spend years of effort and upwards of hundreds of thousands of dollars on primary/secondary school education, and we know that we do a very poor job of making sure that different students have similar education experiences.
So, if SAT scores have low heritability in the US currently, then we would expect that we could figure out which education experiences tend to lead to higher SAT scores and try to do a better job of making sure everyone gets those kinds of experiences. If, on the other hand, heritability is high, then throwing more effort/money at how we do education currently should not be expected to improve SAT scores, and we either need to rethink how we do education, or rethink whether SAT scores measure what we want.
If, on the other hand, heritability is high, then throwing more effort/money at how we do education currently should not be expected to improve SAT scores
I agree with spkoc that this conclusion doesn’t necessarily follow from high heritability. I think it would follow from high and stable heritability across multiple attempted interventions.
An exaggerated story for the point I’m about to make: imagine you’ve never tried to improve SAT scores, and you measure the heritability. You find that, in this particular environment, genetic variance explains 100% of SAT scores. You can predict someone’s SAT scores perfectly just by looking at their genome. You decide to take the half of the population with the highest predicted scores, and keep the SAT a secret from them until the day they take the test. And for the lower half, you give them dedicated tutors to help them prepare. Given the 100% heritability, you expect scores to stay exactly the same. But wait! What no one told you was that before your intervention, the learning environment had been magically uniform for every student. There had been no environmental variance at all, and so the only thing left to explain test scores was genetics. What you didn’t realize is that your heritability estimate gave you no information at all about how environmental changes would affect scores—because there was no environmental variance at all!
A single heritability measurement only tells you, roughly, the ratio of “[existing environmental variance] times [sensitivity to environmental variance]” to “[existing genetic variance] times [sensitivity to genetic variance]”. But it doesn’t do anything to disentangle the sensitivities-to-variances from the actual variances. What if there’s practically zero variance in the environment, but a high sensitivity of the trait you’re looking at to environmental variance? You’d find heritability is very high, but changes to the environment will cause large decreases to heritability. Same thing with genes: what if your trait is 100% determined by genes, but it just so happens that everyone has the exact same genes? You’d find that genetic variance explains zero percent of your trait, but if you then tried some genetic engineering, you’d find heritability shoot upward.
In order to disentangle the “sensitivity of X to environmental variance” from “the level of environmental variance”, you’d have to run multiple interventions over time, and measure the heritability of X after each one (or be confident that your existing environment has lots of variance).
True, but “high and stable heritability” across hundreds (perhaps thousands) of attempted interventions is a pretty good description of the real-world results of education research and practice. See Freddie DeBoer’s “Education Doesn’t Work” for a brief treatment or Kathryn Paige Harden’s The Genetic Lottery for a book-length version.
The evidence indicates that throwing more effort/money at how we do education does not improve IQ scores (for which SAT scores are a thinly-veiled proxy, except that every decade or so they make cosmetic changes to the SAT methodology) or student outcomes. Attempts to rethink education have failed. And IQ is generally useful enough that it is strongly correlated with outcomes we want.
If you’re used to the tech sector with rapid change every decade, moving into the human services sector is going to be a very depressing experience. The low-hanging fruit was picked centuries ago and there hasn’t been any real progress in the last 50 years.
Or at least the particular set of reforms discussed in that article has failed? Even within the context of the US, there do seem to be occasional educational interventions that work, e.g.:
In a state once notorious for its low reading scores, the Mississippi state legislature passed new literacy standards in 2013. Since then Mississippi has seen remarkable gains. Its fourth graders have moved from 49th (out of 50 states) to 29th on the National Assessment of Educational Progress, a nationwide exam. In 2019 it was the only state to improve its scores. For the first time since measurement began, Mississippi’s pupils are now average readers, a remarkable achievement in such a poor state.
Ms Burk attributes Mississippi’s success to implementing reading methods supported by a body of research known as the science of reading. In 1997 Congress requested the National Institute of Child Health and Human Development and the Department of Education to convene a National Reading Panel to end the “reading wars” and synthesise the evidence. The panel found that phonics, along with explicit instruction in phonemic awareness, fluency and comprehension, worked best.
There have been dozens of stories like that; George W Bush got elected on the strength of his education “reforms”. Long-term experience justifies a strong belief (confidence over 90%) that the results will ultimately turn out to be due to a combination of selection bias (cherry-picking) and test fraud. The links are just examples; I’ve been offhandedly following education research and reform for decades. There’s a lot more evidence where that came from, and it tells a very consistent story.
Education simply isn’t a green field—the space of potential solutions has been fairly thoroughly explored (for the set of non-medical solutions that are broadly consistent with Western values and American cultural practices). If you are active in this space without learning that history, you are almost certain to repeat it.
But isn’t this exactly the mainstream intuition that the OP dissolves? My understanding:
a) Heritability measures don’t seem to make sense for really complex traits like intelligence.
b) Heritability measures are not stable outside the environmental conditions in which they were measured.
For instance, some people have sickle cell anemia, which helps them better survive malaria(but otherwise is slightly harmful). If you measure heritability of infant mortality in environment with malaria and then in environment without malaria you get opposite effects. You get a lighter case of malaria with it, so sickle cell probably positively correlates with intelligence if there’s malaria in the environment.
The findings from the Överkalix cohorts imply that grandparental access to food during their slow growth period can modify diabetes and all-cause mortality in grandchildren. Cardiovascular mortality on the other hand was associated with parental, but not grandparental, nutritional experience. The authors interpret their results11 as “proof-of-principle that a sex-specific male-line transgenerational effect exists in humans”, which they consider likely to be epigenetic rather than genetic, cultural or social. A summary of the Överkalix findings is available in19. Their findings have been discussed in renowned peer-reviewed journals20,21,22,23,24 and are cited over 2,000 times (October 2018).
I have this vague memory about reading an article claiming that grandparents’ access to food influenced their grandchildren’s height, moreso than their immediate childrens’, but I can’t seem to find it. Perhaps nonsense.
It is not that intuition. The mainstream intuition is that a high heritability means that variations in the thing that you’re measuring (height, SAT scores, infant mortality) is primarily affected by genetics and cannot be influenced by the environment. A better framing would say that high heritability means that variations in the thing that you’re measuring are not well explained by existing variation in the environment. To reframe, then, what I was going for above, a high heritibility is an upper bound on how much you can expect to improve a specific measure by playing around with existing policy levers, versus having to think outside the box and develop new policy levers.
So, again, pretend that SAT scores have a high measured variability in the US. (I think it’s actually closer to 50 than either extreme, but I don’t have off-hand an actual example of a measure that we might want to increase/reduce the variance of as a society that does have a high heritability.) We might look at nutrition, SES and education and ask whether we can manipulate these to significantly reduce the variance in height and/or SAT scores. Under the high heritability assumption, the answer is that this is very unlikely for both stats.
Again, this is without saying that high heritability means the existing policy levers are not having an effect right now. If we, as a country, stoppd feeding kids at age 5 and left them to fend for themselves, I would expect the survivors to be both shorter and have lower SAT scores than preceding generations. If we stopped sending kids to school, I would expect average SAT scores to drop and the variance to increase significantly. And nothing about heritability argues against any of that.
It also doesn’t mean that there are no policy levers that could have an effect. I am not biologist enough to know if the application of HGH to children on a large scale would increase average height, but what I can say is that heritability has nothing to say on the question, since it is currently an environmental variable with almost no variance itself.
Oops. Accidentally dropped a paragraph about whether you could increase height/SAT scores across the board with the existing policy levers. In that case, a high heritability doesn’t directly say that that’s unlikely, but you would at least have to expect to increase your policy setting to the point where most people are getting more of the nutrition or education or whatever than is currently a few standard deviations above average, since otherwise if less than that much of your policy was having an effect, you would expect to already see it in the heritability measurement. So heritability doesn’t say that feeding kids 10,000 calories a day or sending them to school for 12 hours a day won’t have effects on height/SAT scores, but as a general rule you’d be looking at such extreme interventions that it’s probably not a good direction to go.
I’m still so confused(through no fault of your own, I think you’re right, it just doesn’t fit in my head). Let me try to walk through my thought process.
I assume heritability of SAT score is probably different if you sample across USA, or just upper-mid class suburbs or just South Side Chicago, or just rural Eastern Europe, or just Malawi during a famine. Right? Given that environments are pretty radically different.
What heritability score are we using to determine if policy interventions matter or not? Is the first step to make sure that the region we want to improve has an environment that mimics ‘successful’ regions? Heritability would be very high in a homogenous environment(since that’s the only variation), but it goes down as more varied environments are added to the sample. Heritability is very high if we just look at rich area USA schools, lower if you sample all USA and even lower if we sample the whole world?
Also how is this linked to amplitude of effect? Super high heritability of SAT/IQ in say homogenous Denmark, but presumably the actual variation in scores is lower than in a global sample. Is there a way to say genetics account for +/- 5 points of IQ? So if you’re measuring IQs of 95-105 in your area that’s probably all genetic effects and policy interventions can’t do much?
Edit. I realize now that this is mostly Insub’s point below, but less coherent.
Pretty much. If an intervention is well outside of the set of experiences of your population, there’s probably a reason for that. Perhaps it’s just too new, but it’s likely that it’s inconsistent with the way the culture usually functions (its values as actually implemented) and/or has fairly obvious side effects.
The commenter you are replying to is right. Heritability shows how much of the current variation is genetic in origin, which shows whether the variability of nurture matters within that particular culture. If you want to successfully intervene, it needs to be either in something not very heritable, or with an intervention that is not a significant part of the current landscape of the society. (The fact that outside of context interventions may exist means heritability isn’t a measure of how genetic it is, but that wasn’t Sleeps Furiously’s point.)
Edit: Note, Sleeps Furiously had not replied to spkoc when I saw this and I wrote my response without refreshing.
It can tell you something about existing interventions in a variable. In the US, for instance, we spend years of effort and upwards of hundreds of thousands of dollars on primary/secondary school education, and we know that we do a very poor job of making sure that different students have similar education experiences.
So, if SAT scores have low heritability in the US currently, then we would expect that we could figure out which education experiences tend to lead to higher SAT scores and try to do a better job of making sure everyone gets those kinds of experiences. If, on the other hand, heritability is high, then throwing more effort/money at how we do education currently should not be expected to improve SAT scores, and we either need to rethink how we do education, or rethink whether SAT scores measure what we want.
I agree with spkoc that this conclusion doesn’t necessarily follow from high heritability. I think it would follow from high and stable heritability across multiple attempted interventions.
An exaggerated story for the point I’m about to make: imagine you’ve never tried to improve SAT scores, and you measure the heritability. You find that, in this particular environment, genetic variance explains 100% of SAT scores. You can predict someone’s SAT scores perfectly just by looking at their genome. You decide to take the half of the population with the highest predicted scores, and keep the SAT a secret from them until the day they take the test. And for the lower half, you give them dedicated tutors to help them prepare. Given the 100% heritability, you expect scores to stay exactly the same. But wait! What no one told you was that before your intervention, the learning environment had been magically uniform for every student. There had been no environmental variance at all, and so the only thing left to explain test scores was genetics. What you didn’t realize is that your heritability estimate gave you no information at all about how environmental changes would affect scores—because there was no environmental variance at all!
A single heritability measurement only tells you, roughly, the ratio of “[existing environmental variance] times [sensitivity to environmental variance]” to “[existing genetic variance] times [sensitivity to genetic variance]”. But it doesn’t do anything to disentangle the sensitivities-to-variances from the actual variances. What if there’s practically zero variance in the environment, but a high sensitivity of the trait you’re looking at to environmental variance? You’d find heritability is very high, but changes to the environment will cause large decreases to heritability. Same thing with genes: what if your trait is 100% determined by genes, but it just so happens that everyone has the exact same genes? You’d find that genetic variance explains zero percent of your trait, but if you then tried some genetic engineering, you’d find heritability shoot upward.
In order to disentangle the “sensitivity of X to environmental variance” from “the level of environmental variance”, you’d have to run multiple interventions over time, and measure the heritability of X after each one (or be confident that your existing environment has lots of variance).
True, but “high and stable heritability” across hundreds (perhaps thousands) of attempted interventions is a pretty good description of the real-world results of education research and practice. See Freddie DeBoer’s “Education Doesn’t Work” for a brief treatment or Kathryn Paige Harden’s The Genetic Lottery for a book-length version.
The evidence indicates that throwing more effort/money at how we do education does not improve IQ scores (for which SAT scores are a thinly-veiled proxy, except that every decade or so they make cosmetic changes to the SAT methodology) or student outcomes. Attempts to rethink education have failed. And IQ is generally useful enough that it is strongly correlated with outcomes we want.
If you’re used to the tech sector with rapid change every decade, moving into the human services sector is going to be a very depressing experience. The low-hanging fruit was picked centuries ago and there hasn’t been any real progress in the last 50 years.
Or at least the particular set of reforms discussed in that article has failed? Even within the context of the US, there do seem to be occasional educational interventions that work, e.g.:
There have been dozens of stories like that; George W Bush got elected on the strength of his education “reforms”. Long-term experience justifies a strong belief (confidence over 90%) that the results will ultimately turn out to be due to a combination of selection bias (cherry-picking) and test fraud. The links are just examples; I’ve been offhandedly following education research and reform for decades. There’s a lot more evidence where that came from, and it tells a very consistent story.
Education simply isn’t a green field—the space of potential solutions has been fairly thoroughly explored (for the set of non-medical solutions that are broadly consistent with Western values and American cultural practices). If you are active in this space without learning that history, you are almost certain to repeat it.
But isn’t this exactly the mainstream intuition that the OP dissolves? My understanding:
a) Heritability measures don’t seem to make sense for really complex traits like intelligence.
b) Heritability measures are not stable outside the environmental conditions in which they were measured.
For instance, some people have sickle cell anemia, which helps them better survive malaria(but otherwise is slightly harmful). If you measure heritability of infant mortality in environment with malaria and then in environment without malaria you get opposite effects. You get a lighter case of malaria with it, so sickle cell probably positively correlates with intelligence if there’s malaria in the environment.
There’s also weird epigenetic multigenerational effects: https://www.nature.com/articles/s41467-018-07617-9 (quote is about a reference in this study)
I have this vague memory about reading an article claiming that grandparents’ access to food influenced their grandchildren’s height, moreso than their immediate childrens’, but I can’t seem to find it. Perhaps nonsense.
It is not that intuition. The mainstream intuition is that a high heritability means that variations in the thing that you’re measuring (height, SAT scores, infant mortality) is primarily affected by genetics and cannot be influenced by the environment. A better framing would say that high heritability means that variations in the thing that you’re measuring are not well explained by existing variation in the environment. To reframe, then, what I was going for above, a high heritibility is an upper bound on how much you can expect to improve a specific measure by playing around with existing policy levers, versus having to think outside the box and develop new policy levers.
So, again, pretend that SAT scores have a high measured variability in the US. (I think it’s actually closer to 50 than either extreme, but I don’t have off-hand an actual example of a measure that we might want to increase/reduce the variance of as a society that does have a high heritability.) We might look at nutrition, SES and education and ask whether we can manipulate these to significantly reduce the variance in height and/or SAT scores. Under the high heritability assumption, the answer is that this is very unlikely for both stats.
Again, this is without saying that high heritability means the existing policy levers are not having an effect right now. If we, as a country, stoppd feeding kids at age 5 and left them to fend for themselves, I would expect the survivors to be both shorter and have lower SAT scores than preceding generations. If we stopped sending kids to school, I would expect average SAT scores to drop and the variance to increase significantly. And nothing about heritability argues against any of that.
It also doesn’t mean that there are no policy levers that could have an effect. I am not biologist enough to know if the application of HGH to children on a large scale would increase average height, but what I can say is that heritability has nothing to say on the question, since it is currently an environmental variable with almost no variance itself.
Oops. Accidentally dropped a paragraph about whether you could increase height/SAT scores across the board with the existing policy levers. In that case, a high heritability doesn’t directly say that that’s unlikely, but you would at least have to expect to increase your policy setting to the point where most people are getting more of the nutrition or education or whatever than is currently a few standard deviations above average, since otherwise if less than that much of your policy was having an effect, you would expect to already see it in the heritability measurement. So heritability doesn’t say that feeding kids 10,000 calories a day or sending them to school for 12 hours a day won’t have effects on height/SAT scores, but as a general rule you’d be looking at such extreme interventions that it’s probably not a good direction to go.
I’m still so confused(through no fault of your own, I think you’re right, it just doesn’t fit in my head). Let me try to walk through my thought process.
I assume heritability of SAT score is probably different if you sample across USA, or just upper-mid class suburbs or just South Side Chicago, or just rural Eastern Europe, or just Malawi during a famine. Right? Given that environments are pretty radically different.
What heritability score are we using to determine if policy interventions matter or not? Is the first step to make sure that the region we want to improve has an environment that mimics ‘successful’ regions? Heritability would be very high in a homogenous environment(since that’s the only variation), but it goes down as more varied environments are added to the sample. Heritability is very high if we just look at rich area USA schools, lower if you sample all USA and even lower if we sample the whole world?
Also how is this linked to amplitude of effect? Super high heritability of SAT/IQ in say homogenous Denmark, but presumably the actual variation in scores is lower than in a global sample. Is there a way to say genetics account for +/- 5 points of IQ? So if you’re measuring IQs of 95-105 in your area that’s probably all genetic effects and policy interventions can’t do much?
Edit. I realize now that this is mostly Insub’s point below, but less coherent.
Pretty much. If an intervention is well outside of the set of experiences of your population, there’s probably a reason for that. Perhaps it’s just too new, but it’s likely that it’s inconsistent with the way the culture usually functions (its values as actually implemented) and/or has fairly obvious side effects.
The commenter you are replying to is right. Heritability shows how much of the current variation is genetic in origin, which shows whether the variability of nurture matters within that particular culture. If you want to successfully intervene, it needs to be either in something not very heritable, or with an intervention that is not a significant part of the current landscape of the society. (The fact that outside of context interventions may exist means heritability isn’t a measure of how genetic it is, but that wasn’t Sleeps Furiously’s point.)
Edit: Note, Sleeps Furiously had not replied to spkoc when I saw this and I wrote my response without refreshing.