Infertility rates are rising and nobody seems to quite know why. Below is what feels like a possible (trivial) explanation that I haven’t seen mentioned anywhere.
a family is either fertile or infertile, and fertility is hereditary
the modal fertile family can have up to 10 kids, the modal infertile family can only have 2 kids
in the olden days families aimed to have as many kids as they could
now families aim to have 2 kids each
Under this model, in the olden days we would find a high proportion of fertile people in the gene pool, but in the modern world we wouldn’t. Put differently, the old convention lead to a strong positive correlation between fertility and participation in the gene pool, and the new convention leads to 0 correlation. This removes the selective pressure on fertility, hence we should expect fertility to drop / infertility to rise.
Empirical evidence for this would be something like an analysis of the time series of family size variance and infertility—is lower variance followed by increased infertility?
Regarding female fertility, this report from Norway outlines the trend that I vaguely thought was representative of most of the developed world over the last 100 years.
Female fertility is trickier to measure, since female fertility and age are strongly correlated, and women have been having kids later, so it’s important (and likely tricky) to disentangle this confounder from the data.
Reflecting on the particular ways that perfectionism differs from the optimal policy (as someone who suffers from perfectionism) and looking to come up with simple definitions, I thought of this:
perfectionism looks to minimize the distance between an action and the ex-post optimal action but heavily dampening this penalty for the particular action “do nothing”
optimal policy says to pick the best ex-ante action out of the set of all possible actions, which set includes “do nothing”
So, perfectionism will be maximally costly in an environment where you have lots of valuable options of new things you could do (breaking from status quo) but you’re unsure whether you can come close to the best one, like you might end up choosing something that’s half as good as the best you could have done. Optimal policy would say to just give it your best, and that you should be happy since this is an amazingly good problem to have, whereas perfectionism will whisper in your ear how painful it might be to only get half of this very large chunk of potential utility, and wouldn’t it be easier if you just waited.
Regularization implements Occam’s Razor for machine learning systems.
When we have multiple hypotheses consistent with the same data (an overdetermined problem) Occam’s Razor says that the “simplest” one is more likely true.
When an overparameterized LLM is traversing the subspace of parameters that solve the training set seeking the smallest l2-norm say, it’s also effectively choosing the “simplest” solution from the solution set, where “simple” is defined as lower parameter norm i.e. more “concisely” expressed.
Unfortunately the entire complexity has just been pushed one level down into the definition of “simple”. The L2 norm can’t really be what we mean by simple, because simply scaling the weights in a layer by A, and the weights in the next layer by 1/A leaves the output of the network invariant, assuming ReLU activations, yet you can obtain arbitrarily high L2 norms by just choosing A high enough.
Agreed with your example, and I think that just means that L2 norm is not a pure implementation of what we mean by “simple”, in that it also induces some other preferences. In other words, it does other work too. Nevertheless, it would point us in the right direction frequently e.g. it will dislike networks whose parameters perform large offsetting operations, akin to mental frameworks or beliefs that require unecessarily and reducible artifice or intermediate steps.
Worth keeping in mind that “simple” is not clearly defined in the general case (forget about machine learning). I’m sure lots has been written about this idea, including here.
I wonder how much of the tremendously rapid progress of computer science in the last decade owes itself to structurally more rapid truth-finding, enabled by:
the virtual nature of the majority of the experiments, making them easily replicable
the proliferation of services like github, making it very easy to replicate others’ experiments
(a combination of the points above) the expectation that one would make one’s experiments easily available for replication by others
There are other reasons to expect rapid progress in CS (compared to, say, electrical engineering) but I wonder how much is explained by this replication dynamic.
Very little, because most CS experiments are not in fact replicable (and that’s usually only one of several serious methodological problems).
CS does seem somewhat ahead of other fields I’ve worked in, but I’d attribute that to the mostly-separate open source community rather than academia per se.
To be sure, let’s say we’re talking about something like “the entirety of published material” rather than the subset of it that comes from academia. This is meant to very much include the open source community.
Very curious, in what way are most CS experiments not replicable? From what I’ve seen in deep learning, for instance, it’s standard practice to include a working github repo along with the paper (I’m sure you know lots more about this than I do). This is not the case in economics, for instance, just to pick a field I’m familiar with.
Fuzzing is a generally pretty healthy subfield, but even there most peer-reviewed papers in top venues are still are completely useless! Importantly, “a ‘working’ github repo” is really not enough to ensure that your results are reproducible, let alone ensure external validity.
It feels like (at least in the West) the majority of our ideation about the future is negative, e.g.
popular video games like Fallout
zombie apocalypse themed tv
shows like Black Mirror (there’s no equivalent White Mirror)
Are we at a historically negative point in the balance of “good vs bad ideation about the future” or is this type of collective pessimistic ideation normal?
If the balance towards pessimism is typical, is the promise of salvation in the afterlife in e.g. Christianity a rare example of a powerful and salient positive ideation about our futures (conditioned on some behavior)?
I agree. I feel like this is a very recent change as well. We used to be hopeful about the future, creating sci-fi about utopias rather than writing nightmare scenarios.
The west is becoming less self-affirming over time, and our mental health is generally getting worse. I think it’s because of historic guilt, as well as a kind of self-loathing pretending that it’s virtue (anti-borders, anti-nationalism, anti-natalism) not to mention the slander of psychological drives which strive for growth and quality (competition, hierarchies, ambition, elitism, discrimination/selection/gatekeeping)
I do not believe that the salvation in the afterlife is the opposite of this, but rather the same. It ultimately talks negatively about life and actual reality, comparing it to some unreachable ideal. It’s both pessimistic, as well as a psychological cope which makes it possible to endure this pessimism. The message is something akin to “Endure, and you will be rewarded in the end”
It’s a weariness we will have to overcome. I feel like our excessive tendency to problem-solving has caused us to view life as a big collection of problems, rather than something which is merely good but imperfect
From personal observation, kids learn text (say, from a children’s book, and from songs) back-to-front. That is, the adult will say all but the last word in the sentence, and the kid will (eventually) learn to chime in to complete the sentence.
This feels correlated to LLMs learning well when tasked with next-token prediction, and those predictions being stronger (less uniform over the vocabulary) when the preceding sequences get longer.
I wonder if there’s a connection to having rhyme “live” in the last sound of each line, as opposed to the first.
A lot of memory seems to be linear, possibly because most information in the world is encoded linearly. If I was to tell you the 20th letter of the alphabet, I’d have to go through every letter it in my head. It’s a linked-list data structure.
Even many memory techniques, like the mind palace, is ordered, with each item linking to the next.
I don’t think this is the same as markov-chains or predicting the next item, but that it has to do with the most common data structure of information being linear.
As for making the first word rhyme instead of the last, that’s an interesting thought! I actually have no idea. When I rhyme like that in my head, it sounds wrong, but I couldn’t tell you the reason. You may be on to something.
Is meditation provably more effective than “forcing yourself to do nothing”?
Much like sleep is super important for good cognitive (and, of course, physical) functioning, it’s plausible that waking periods of not being stimulated (i.e. of boredom) are very useful for unlocking increased cognitive performance. Personally I’ve found that if I go a long time without allowing myself to be bored, e.g. by listening to podcasts or audiobooks whenever I’m in transition between activities, I’m less energetic, creative, sharp, etc.
The problem is that as a prescription “do nothing for 30 minutes” would be rejected as unappealing by most. So instead of “do nothing” it’s couched as “do this other thing” with a focus on breathing and so on. Does any of that stuff actually matter or does the benefit just come from doing nothing?
I think what those other things do is help you reach that state more easily and reliably. It’s like a ritual that you do before the actual task, to get yourself into the right frame of mind and form a better connection, similar to athletes having pre game rituals.
Also yeah, I think it makes the boredom easier to manage and helps you slowly get into it, rather than being pushed into it without reference.
Probably a lot of other hidden benefits though, because most meditation practices have been optimized for hundreds of years, and are better than others for a reason.
The parallel to athlete pre game rituals is an interesting one, but I guess I’d be interested in seeing the comparison between the following two groups:
group A: is told to meditate the usual way for 30 minutes / day, and does
group B: is told to just sit there for 30 minutes / day, and does
So both of the groups considered are sitting quietly for 30 minutes, but one group is meditating while the other is just sitting there. In this comparison, we’d be explicitly ignoring the benefit from meditation which acts via the channel of just making it more likely you actually sit there quietly for 30 minutes.
Infertility rates are rising and nobody seems to quite know why. Below is what feels like a possible (trivial) explanation that I haven’t seen mentioned anywhere.
I’m not in this field personally so it’s possible this theory is out there, but asking GPT about it doesn’t yield the proposed explanation: https://chat.openai.com/share/ab4138f6-978c-445a-9228-674ffa5584ea
Toy model:
a family is either fertile or infertile, and fertility is hereditary
the modal fertile family can have up to 10 kids, the modal infertile family can only have 2 kids
in the olden days families aimed to have as many kids as they could
now families aim to have 2 kids each
Under this model, in the olden days we would find a high proportion of fertile people in the gene pool, but in the modern world we wouldn’t. Put differently, the old convention lead to a strong positive correlation between fertility and participation in the gene pool, and the new convention leads to 0 correlation. This removes the selective pressure on fertility, hence we should expect fertility to drop / infertility to rise.
Empirical evidence for this would be something like an analysis of the time series of family size variance and infertility—is lower variance followed by increased infertility?
How robust is the information that infertility rates are rising?
To be sure, I’m not an expert on the topic.
Declines in male fertility I think are regarded as real, though I haven’t examined the primary sources.
Regarding female fertility, this report from Norway outlines the trend that I vaguely thought was representative of most of the developed world over the last 100 years.
Female fertility is trickier to measure, since female fertility and age are strongly correlated, and women have been having kids later, so it’s important (and likely tricky) to disentangle this confounder from the data.
Reflecting on the particular ways that perfectionism differs from the optimal policy (as someone who suffers from perfectionism) and looking to come up with simple definitions, I thought of this:
perfectionism looks to minimize the distance between an action and the ex-post optimal action but heavily dampening this penalty for the particular action “do nothing”
optimal policy says to pick the best ex-ante action out of the set of all possible actions, which set includes “do nothing”
So, perfectionism will be maximally costly in an environment where you have lots of valuable options of new things you could do (breaking from status quo) but you’re unsure whether you can come close to the best one, like you might end up choosing something that’s half as good as the best you could have done. Optimal policy would say to just give it your best, and that you should be happy since this is an amazingly good problem to have, whereas perfectionism will whisper in your ear how painful it might be to only get half of this very large chunk of potential utility, and wouldn’t it be easier if you just waited.
Regularization implements Occam’s Razor for machine learning systems.
When we have multiple hypotheses consistent with the same data (an overdetermined problem) Occam’s Razor says that the “simplest” one is more likely true.
When an overparameterized LLM is traversing the subspace of parameters that solve the training set seeking the smallest l2-norm say, it’s also effectively choosing the “simplest” solution from the solution set, where “simple” is defined as lower parameter norm i.e. more “concisely” expressed.
Unfortunately the entire complexity has just been pushed one level down into the definition of “simple”. The L2 norm can’t really be what we mean by simple, because simply scaling the weights in a layer by A, and the weights in the next layer by 1/A leaves the output of the network invariant, assuming ReLU activations, yet you can obtain arbitrarily high L2 norms by just choosing A high enough.
Agreed with your example, and I think that just means that L2 norm is not a pure implementation of what we mean by “simple”, in that it also induces some other preferences. In other words, it does other work too. Nevertheless, it would point us in the right direction frequently e.g. it will dislike networks whose parameters perform large offsetting operations, akin to mental frameworks or beliefs that require unecessarily and reducible artifice or intermediate steps.
Worth keeping in mind that “simple” is not clearly defined in the general case (forget about machine learning). I’m sure lots has been written about this idea, including here.
I wonder how much of the tremendously rapid progress of computer science in the last decade owes itself to structurally more rapid truth-finding, enabled by:
the virtual nature of the majority of the experiments, making them easily replicable
the proliferation of services like github, making it very easy to replicate others’ experiments
(a combination of the points above) the expectation that one would make one’s experiments easily available for replication by others
There are other reasons to expect rapid progress in CS (compared to, say, electrical engineering) but I wonder how much is explained by this replication dynamic.
Very little, because most CS experiments are not in fact replicable (and that’s usually only one of several serious methodological problems).
CS does seem somewhat ahead of other fields I’ve worked in, but I’d attribute that to the mostly-separate open source community rather than academia per se.
To be sure, let’s say we’re talking about something like “the entirety of published material” rather than the subset of it that comes from academia. This is meant to very much include the open source community.
Very curious, in what way are most CS experiments not replicable? From what I’ve seen in deep learning, for instance, it’s standard practice to include a working github repo along with the paper (I’m sure you know lots more about this than I do). This is not the case in economics, for instance, just to pick a field I’m familiar with.
See e.g. https://mschloegel.me/paper/schloegel2024sokfuzzevals.pdf
Fuzzing is a generally pretty healthy subfield, but even there most peer-reviewed papers in top venues are still are completely useless! Importantly, “a ‘working’ github repo” is really not enough to ensure that your results are reproducible, let alone ensure external validity.
It feels like (at least in the West) the majority of our ideation about the future is negative, e.g.
popular video games like Fallout
zombie apocalypse themed tv
shows like Black Mirror (there’s no equivalent White Mirror)
Are we at a historically negative point in the balance of “good vs bad ideation about the future” or is this type of collective pessimistic ideation normal?
If the balance towards pessimism is typical, is the promise of salvation in the afterlife in e.g. Christianity a rare example of a powerful and salient positive ideation about our futures (conditioned on some behavior)?
I agree. I feel like this is a very recent change as well. We used to be hopeful about the future, creating sci-fi about utopias rather than writing nightmare scenarios.
The west is becoming less self-affirming over time, and our mental health is generally getting worse. I think it’s because of historic guilt, as well as a kind of self-loathing pretending that it’s virtue (anti-borders, anti-nationalism, anti-natalism) not to mention the slander of psychological drives which strive for growth and quality (competition, hierarchies, ambition, elitism, discrimination/selection/gatekeeping)
I do not believe that the salvation in the afterlife is the opposite of this, but rather the same. It ultimately talks negatively about life and actual reality, comparing it to some unreachable ideal. It’s both pessimistic, as well as a psychological cope which makes it possible to endure this pessimism. The message is something akin to “Endure, and you will be rewarded in the end”
It’s a weariness we will have to overcome. I feel like our excessive tendency to problem-solving has caused us to view life as a big collection of problems, rather than something which is merely good but imperfect
From personal observation, kids learn text (say, from a children’s book, and from songs) back-to-front. That is, the adult will say all but the last word in the sentence, and the kid will (eventually) learn to chime in to complete the sentence.
This feels correlated to LLMs learning well when tasked with next-token prediction, and those predictions being stronger (less uniform over the vocabulary) when the preceding sequences get longer.
I wonder if there’s a connection to having rhyme “live” in the last sound of each line, as opposed to the first.
A lot of memory seems to be linear, possibly because most information in the world is encoded linearly. If I was to tell you the 20th letter of the alphabet, I’d have to go through every letter it in my head. It’s a linked-list data structure.
Even many memory techniques, like the mind palace, is ordered, with each item linking to the next.
I don’t think this is the same as markov-chains or predicting the next item, but that it has to do with the most common data structure of information being linear.
As for making the first word rhyme instead of the last, that’s an interesting thought! I actually have no idea. When I rhyme like that in my head, it sounds wrong, but I couldn’t tell you the reason. You may be on to something.
Is meditation provably more effective than “forcing yourself to do nothing”?
Much like sleep is super important for good cognitive (and, of course, physical) functioning, it’s plausible that waking periods of not being stimulated (i.e. of boredom) are very useful for unlocking increased cognitive performance. Personally I’ve found that if I go a long time without allowing myself to be bored, e.g. by listening to podcasts or audiobooks whenever I’m in transition between activities, I’m less energetic, creative, sharp, etc.
The problem is that as a prescription “do nothing for 30 minutes” would be rejected as unappealing by most. So instead of “do nothing” it’s couched as “do this other thing” with a focus on breathing and so on. Does any of that stuff actually matter or does the benefit just come from doing nothing?
There are some styles of meditation that are explicitly described as “just sitting” or “doing nothing.”
Kind of related Quanta article from a few days ago: https://www.quantamagazine.org/what-your-brain-is-doing-when-youre-not-doing-anything-20240205/
I think what those other things do is help you reach that state more easily and reliably. It’s like a ritual that you do before the actual task, to get yourself into the right frame of mind and form a better connection, similar to athletes having pre game rituals.
Also yeah, I think it makes the boredom easier to manage and helps you slowly get into it, rather than being pushed into it without reference.
Probably a lot of other hidden benefits though, because most meditation practices have been optimized for hundreds of years, and are better than others for a reason.
The parallel to athlete pre game rituals is an interesting one, but I guess I’d be interested in seeing the comparison between the following two groups:
group A: is told to meditate the usual way for 30 minutes / day, and does
group B: is told to just sit there for 30 minutes / day, and does
So both of the groups considered are sitting quietly for 30 minutes, but one group is meditating while the other is just sitting there. In this comparison, we’d be explicitly ignoring the benefit from meditation which acts via the channel of just making it more likely you actually sit there quietly for 30 minutes.