foodforthought

Karma: 186

I am a scientist by vocation (and also by profession), in particular a biologist. Entangled with this calling is an equally deep interest in epistemology (specifically the warrant of inductive inference). The kinds of scientific explanations I find satisfying are quantitative (often statistical) models or theories that parsimoniously account for empirical biological observations in normative (functional, teleological) terms.

My degrees (BS, PhD) are in biology but my background is interdisciplinary, including also philosophy, psychology, mathematics/statistics, and computer science/machine learning. For the last few decades I have been researching the neurobiology of sensory perception, decision-making, and value-based choice in animals (including humans) and in models.

Now and then I get fascinated by something that isn’t obviously related to my existing domain, sometimes leading to semi-expertise and/or semi-professional activity in seemingly random other domains, and occasionally leading to an outright career change. At the moment that topic I’m fascinated by is AI-alignment, which brings me here.

I consider myself a visitor to your forum, in that my context is mainly from without.

Social media are anathema to me, but this forum seems to be an outlier.

foodforthought 29 Nov 2025 23:21 UTC
1 point
0
on: Abstract advice to researchers tackling the difficult core problems of AGI alignment
I get excited about the possibility of contributing to AI alignment research whenever people talk about the problem being hard in this particular way. The problems are still illegible. The field needs relentless Original Seeing. Every assumption needs to be Truly Doubted. The approaches that will turn out to be fruitful probably haven’t even been imagined yet. It will be important to be able to learn and defer while simultaneously questioning, to come up with bold ideas while not being attached to them. It will require knowing your stuff, yet it might be optimal not to know too much about what other people have thought so far (“I would suggest not reading very much more.”). It’ll be important for people attempting this to be exceptionally good at rigorous, original conceptual thinking, thinking about thinking, grappling with minds, zooming all the way in and all the way out, constantly. It’ll probably require making ego and career sacrifices. Bring it on!
However, here’s an observation: the genuine potential to make such a contribution is itself highly illegible. Not only to the field, but perhaps to the potential contributor as well.
Apparently a lot of people fancy themselves to have Big Ideas or to be great at big-picture thinking, and most of them aren’t nearly as useful as they think they are. I feel like I’ve seen that sentiment many times on LW, and I’m guessing that’s behind statements like:
It’s not remotely sufficient, and is often anti-helpful, to just be like “Wait, actually what even is alignment? Alignment to what?”.
or the Talent Needs post that said
Finally, that so few people are expressing interest in hiring Connectors, relative to the apparent high numbers of aspiring Connectors applying to MATS and other field-building programs, tells us that the ecosystem is potentially attracting an excess of inexperienced Connectors who may not be adequately equipped to refine their ideas...
This presents a bit of a paradox. Suppose there exist a few rare, high-potential contributors not already working in AI who would be willing and able to take up the challenge you describe. It seems like the only way they could make themselves sufficiently legibly useful would be to work their way up through the ranks of much less abstract technical AI research until they distinguish themselves. That’s likely to deter horizontal movement of mature talent from other disciplines. I’m curious if you think this is true; or if you think starting out in object-level technical research is necessary training/preparation for the kind of big-picture work you have in mind; or if you think there are other routes of entry.

foodforthought 28 Nov 2025 2:50 UTC
2 points
0
on: A Thanksgiving Memory
Hopefully implied by omission, but I probably should have said that nobody was badly hurt; the workers all got out (one tried to go back in to rescue our parakeet, but the firefighters wouldn’t let her).

foodforthought 15 Nov 2025 1:54 UTC
5 points
0
on: The rare, deadly virus lurking in the Southwest US, and the bigger picture
fwiw when I was doing research on wild-caught california ground squirrels I was told the population was known to harbor hantavirus (as well as plague). Consequently, we had to suit up like an anthrax research facility every time we worked with them. I have no idea why this was believed.

foodforthought 9 Nov 2025 23:34 UTC
24 points
0
on: One Shot Singalonging is an attitude, not a skill or a song-difficulty-level*
Completely agree with your observations, and I say this as someone who (a) grew up with family singing, campfire singing (b) had pretty extensive choral training; (c) nevertheless later participated in and led community singing groups built on the idea that it’s totally, absolutely fine to sing “badly”; (d) for many years hosted a successful wassailing party at which most people were not Christian and not familiar with caroling, yet happily and credibly belted out the Christmas carols they had just learned, on the porches of surprised neighbors; (e) is now a folkie very active in the pub singing tradition (f) with an armchair interest in ethnomusicology, oral tradition, and the neurobiosociology of community singing.
SO… a few thoughts:
You are absolutely right that the most important factor is giving people permission to sing, that everyone has the right to sing, your voice doesn’t have to be good, you don’t have to be in tune, and in fact it will be fine. If there are professional musicians present it can be important to explain to them what is going on, why they should be happy to hear bad singers sing, and how they can help by singing the melody loudly, and not wincing.
You may find helpful resources at https://singout.org/communitysings/, look up Pete Seeger’s Tone Deaf Choir (historical), Matt Watroba’s Community Sings (current, I think). They were brilliant at getting big crowds of non-singers to sing (and surprise themselves with how good they sound).
I’d be happy to exchange notes on repertoire. I am not sure what themes exactly suit the Rationalist Solstice scene, but I know what worked well for my motley wassailing crew and my community singing group.
There are characteristics of songs from oral tradition that support/encourage everyone to sing, which are common in older traditional songs, religious/church songs, childrens/camp songs. Yes, they arose from contexts where the participants could not read, but this is irrelevant. You do not want people reading words off of a sheet of paper or their phone. You want them to be present to the room and just sing. You want it to be easy to sing in the pitch dark with a candle in one hand and a glass of grog in the other. So the same rules apply.
Predictability. The tune is repetitive (no modulations, bridges, etc), and the lyrics have a formulaic pattern. A good example: “Where have all the flowers gone?”. If someone has never heard the song, they have to stop singing and listen to hear the one word that is new in each verse; but then they can predict how the entire next verse will go, and can sing along to the whole verse as well as the chorus. And that song is not silly, and not a bad candidate.
Repetition. Songs with a chorus, as noted, you can sing the chorus once at the outset, and then people can sing it every time it repeats. Pro tip: sing every verse, and repeat the chorus after every verse. Stage folk performers will skip verses and only sometimes sing the chorus to avoid boring the audience; but it’s not boring when there is no audience and everyone is singing. But choruses aren’t the only form of this. Some songs have call and response where you repeat each line (or there’s a formula for the response to the called line). Some have a refrain in which the last line or two of each verse is sung again. Look for songs with these features.
Familiarity. Obvious one. If a lot of people recognize a song it helps, even if they just hum the tune. A good example of this might be Silent Night. In traditional music, including church hymnals and pub ballads, tunes are heavily re-used. The same tunes are re-used for many different sets of lyrics, so you can leverage that everyone already knows the tune. Parodies (writing new songs to well known tunes) work well for this reason.
Physicality. Clapping, stomping, snapping, whatever, lets people participate even if they don’t know the words or tune—and surprisingly lowers inhibitions for singing.
For totally novel songs I think the best you can do is have an optional pre-run for people who want to learn them, use the same ones year after year, and make it ok to just listen and enjoy the ones you don’t know. People pick up songs up with remarkable ease. They will accidentally find they are singing it next time. In this context, story songs are the easiest for people to remember; our brains are wired for stories. For example ‘Good King Wenceslas’ was the favorite most belted out at my Wassail, even though there are no choruses or refrains.
Since you are taking a long term view on this: wanna co-org a workshop on “singing for people who can’t sing” at LessOnline next year?

foodforthought 9 Nov 2025 16:22 UTC
−2 points
−3
on: Trying to understand my own cognitive edge
It seems impractical to recommend that someone spend a few years in cryptography
That’s too literal. How about: always try to play chess/tennis with someone better than you, when you can. Get your early training—when it’s your full time job to study—in the field you struggle to keep up in, not the one you clearly dominate in. Be the small fish in a big pond. You learn most from people who are better than you. You will learn epistemic humility and self-skepticism. You hopefully learn not to stake your ego on your genius. you learn how to make yourself useful to people you want to learn from.
(But don’t pick the field you can’t keep up in at all; you won’t learn anything when you are utterly lost; you’ll likely be demoralized by the attempt, even if everyone is kind; and you risk being a true burden, toward whom kindness won’t always be extended).

foodforthought 9 Nov 2025 16:11 UTC
1 point
0
on: Trying to understand my own cognitive edge
Are there others who could make a similar claim of having exceptionally good and hard to explain intuitions
Yes. PM me.

foodforthought 9 Nov 2025 16:11 UTC
1 point
0
on: Trying to understand my own cognitive edge
Not only is it rare but there seems to be a surprisingly large gap between my intuitions and the next closest person’s.
This is a natural consequence of having many interests and switching fields many times; you bring a unique set of factual knowledge, concepts, heuristics, theoretical frameworks or insights to the problem at hand. You pay the cost of not likely being the most-expert or most-competent in most or all the fields you care about, but you reap the benefit of having uncommon insights, which tend to be orthogonal to the rest of the field (and perhaps for this reason incomprehensible to them).
Weak analogy: If reality could be mapped to one orthogonal basis set, it’s as if each specialization only has a few basis functions; the good fields have at least one basis function that explains a large fraction of the power in at least the signal set [dimension of reality] they study. But other parts of reality that don’t happen to project onto their set are entirely invisible to them. Interdisciplinary field-switchers get to accumulate basis functions as they go; so they always have some at their disposal that are orthogonal to the set in use within the current discipline. The more disparate two disciplines, the less overlap in their endogenous basis sets (but perhaps that is a tautological statement, a definition of ‘disparate’).

foodforthought 9 Nov 2025 16:01 UTC
1 point
0
on: Trying to understand my own cognitive edge
It seems hard to explain using anything we know from cognitive science. Standard explanations for good intuitions include that they’re distilled from extensive prior experience or reasoning, but I moved from field to field and as a result was often a newcomer.
This could be exactly why you were able to have exceptionally good intuitions, see https://www.lesswrong.com/posts/Afdohjyt6gESu4ANf/most-people-start-with-the-same-few-bad-ideas?commentId=JBHcLYgk77c7vxeGk

foodforthought 20 Oct 2025 19:44 UTC
12 points
1
on: Humanity Learned Almost Nothing From COVID-19
pertinent analysis from Rand Institute:
https://www.rand.org/pubs/external_publications/EP70594.html
“At the end of the last decade and the beginning of this one, human society itself was subject to a kind of penetration test: COVID-19. The virus, an unthinking adversary, probed the world’s ability to defend against new pathogens. And by the end of the test, it was clear that humanity had failed.”

foodforthought 13 Oct 2025 15:58 UTC
1 point
0
on: Experiment: Test your priors on Bernoulli processes.
To clarify, the ground truth P(R) is constrained to be constant over the 5 trials of any given experiment?

foodforthought 11 Oct 2025 17:45 UTC
5 points
0
on: Examples of Highly Counterfactual Discoveries?
My immediate thought is McClintock’s transposable elements. AFAICT, this has only been mentioned by AI-generated lists in this thread, so to fill in a bit more for anyone who doesn’t know the story: in the 1940s, McClintock observed genetic and cytological evidence from crosses of corn plants, which she argued could best be explained by assuming certain genetic elements routinely change their position in the genetic map, often breaking other genes when they insert, and restoring those genes again when they excise. For context, the discovery that genes had fixed positions on linear genetic maps that were collinear with chromosomes was still relatively new (1913), and the field of genetics was largely consumed by the job of determining these maps. Her interpretation was therefore very much against the current, and it was mostly dismissed and derided. But she was right. It wasn’t until molecular biology confirmed their existence in the 60s-70s that transposable elements (“jumping genes”) became widely accepted. She got the Nobel Prize for her discovery over four decades after she made it.

I take it the reason for asking for such case studies is that singular discoveries can be exceptionally impactful, so it would be good to enrich for them. Therefore it’s of interest to ask what happened to McClintock in the intervening decades. My understanding is that she was able to continue her work the entire time, despite the skepticism of the field, due entirely to the Carnegie Institute. Carnegie Institute created a permanent position at Cold Spring Harbor Lab specifically for her, freeing her from teaching and administrative obligations, but more importantly, shielding her from the need for peer acceptance of her ideas (peer-reviewed grants, peer-reviewed papers). Importantly they backed her permanently and unconditionally, so that she was completely free to pursue whatever drove her curiosity, regardless of anyone else’s opinion, even theirs.
This highlights the huge impact a private benefactor (individual or institution) can have by backing individual innovators. The trick is how to figure out who is worth backing. It’s only impactful if one ignores or even actively anti-correlates with the usual metrics that academia rewards; but some or even most marginalized mavericks are in fact crackpots, so anticorrelating isn’t enough. One has to be confident in positively judging people or ideas to be worthwhile, without relying on evaluations by leaders and experts.

foodforthought 5 Oct 2025 15:50 UTC
2 points
0
in reply to: the gears to ascension’s comment on: a quick thought about AI alignment
useful, thanks

foodforthought 5 Oct 2025 15:32 UTC
2 points
1
in reply to: RussellThor’s comment on: a quick thought about AI alignment
thanks for the term for this and the link

foodforthought 12 Sep 2025 14:12 UTC
1 point
2
on: Contra Shrimp Welfare.
Even if shrimp consciously experience pain or suffering, my question is, why do we think electrocution is more humane than freezing?
The idea of freezing to death sounds horrible because we are warm-blooded animals. In cold blooded animals, cooling gradually slows metabolism inducing torpor. I would have thought for a shrimp, ice slurry immersion would be a gentle method of putting them under anesthesia; sounds like that takes a minute. Do they behaviorally indicate pain or aversion? I’m more surprised that an ice slurry kills them. Apparently pink shrimp (the kind people eat) thrive best in ocean waters between 0-8C and have been found living in −2C waters (wikipedia).

foodforthought 10 Sep 2025 21:45 UTC
4 points
4
on: Obligated to Respond
(I think part of why this goes squirrelly, in practice, is that it’s easy for a certain type of person to feel like they’re engaging in a purely one-on-one interaction, in places like Facebook or Twitter or LessWrong or wherever. Like, if one is already a pays-less-attention-to-the-audience type Pokémon to begin with, then it’s easy for the audience to fall completely out of your thoughts as you tunnel-vision on the person you’re directly responding to. But I sort of can’t ever not-notice the other monkeys watching.)
This post in general, and this comment especially, was helpful in clarifying for me why I hate social media so much. (This forum being an exception). It seems to me that people are much less rational when arguing on social media than they are in a private one-on-one conversations, because they can’t help noticing the other monkeys watching—even if they claim the contrary. This pertains to the obligation-to-respond case and a much wider set of dynamics.
The more aggressive-seeming sorts may honestly believe they are just purists for truth and socially oblivious. For some, this may be true. But more often I notice an interesting pattern: in public online discussions, their arguments are littered with subtle rhetorical devices (argumentum ad hominem, ad populum, ad ridiculum, ad verecundium, etc.) (in english: glib, snarky, pontificating, witty banter, etc.) -- none of which are aimed at helpfully updating their own or the other person’s worldview, and rather seem to be aimed at playing to the audience. Tellingly, this dimension often disappears when same person is in a one-on-one conversation, even on the identical disagreement with the same ‘opponent’. The same person can be much more constructive, rational, curious, open-minded, willing to concede uncertainty, etc. when the other monkeys aren’t watching. It is also vastly more epistemically efficient to communicate, figure out common ground, and distill differences in one-on-one conversations, without the distraction of tracking what the audience might know/think as well.
So I’m a big advocate of this: as soon as people realize they substantively disagree, assuming everyone’s real motivation is to figure out what is actually true (or at least that’s the motivation we all wish to honor), work it out in a one-on-one conversation. You might reach agreement, or reach clarity about the root disagreement. One or the other party might decide the whole question is not that important, or other person isn’t arguing in good faith, or whatever, and can choose to abandon the conversation at any time—without worrying about how that will be perceived. In the end, if either of you think the conversation was constructive, you can always write up a distillation of the useful bits for a wider audience. (Co-authoring a disagreement distillation seems like a genre we should especially encourage).
But while you are doing hard intellectual and perhaps emotional work of wrangling with a disagreement, having an audience is generally not helpful, and often gets in the way.

foodforthought 7 Sep 2025 19:12 UTC
5 points
0
on: foodforthought’s Shortform
PSA for people who are interested in nutrition and health, and frustrated by the level of BS in the media surrounding these topics: I find the Nutrition Diva podcast to be exceptionally objective and rational. It’s my go to place to check for an informed take on any nutrition claim or question. She does the work of looking up and reading the original research articles, checking if the experiment is well-designed, inference valid, and if the data actually support the claims, a task I normally don’t have time for myself. She is unusually clear on epistemic status (for example, distinguishing between evidence of absence and absence of evidence; articulating uncertainty rather than burying it). And she doesn’t seem to shy away from taking unpopular stands, if that’s where the data land.

foodforthought 7 Sep 2025 18:50 UTC
4 points
0
on: Most People Start With The Same Few Bad Ideas
More generally, in the very early phase of discovery in any field, when nobody has any idea what is going to turn out to be true or fruitful, insiders of a field tend to get bogged down because they have an overly correlated set of ideas. Not only the main ideas (like hypotheses, in science) but also concepts, tools, approaches, analogies, aesthetic preferences, and background knowledge. So a vast effort gets allocated to a tiny corner of the potential search space. This is why cross disciplinary transplants can make outsized contributions.
This is in tension with expertise. Amateurs and beginners by definition lack a lot of existing knowledge. This is a blessing, because they don’t “know” things all the experts “know” incorrectly, and a curse because they also don’t “know” things all the experts know correctly. In a decently rigorous and otherwise productive field, most of what experts know is in the latter category^[1].
This leads me to a speculation: the optimal way to tap the potential contributions of field-outsiders is to pair them up with experts, or integrate them into teams of experts. That is a big investment and commitment on both parts, so a prior vetting / recruitment step is needed.
As a first step you can invite such people to participate in high level, big picture conversations on a one time or short term basis. The interaction group has to be big enough that it can accommodate a couple of wild cards, but small enough that it won’t be a huge drag on the experts to have to constantly explain basic things. That said, asking experts to explain things in plain language which they all take for granted as obvious, is often the value added.
The outsider has to be the right sort, though. Smart enough to pick up new ideas quickly; confident enough to ask “dumb” questions or speak up in general; articulate enough to explain their ideas to people outside their own domain of expertise; enough social intelligence to notice when it’s a good time to pipe up vs be quiet, and capacity to self-regulate accordingly. And it takes a certain kind of creativity to be good at recognizing unseen connections or implications.
It isn’t necessarily obvious which other disciplines have the sauce that is missing. Physics and Philosophy are often good bets. But here’s a speculation: anyone who is a seasoned expert in any completely different but rigorous and successful domain is a good bet. (Young folks with a few years in another strong discipline also make great trainees. People who are long-established in a field that is mostly bankrupt are less likely to help than any random person on the street).
So if you are running a workshop or conference or symposium that is not too big, where many of the participants are high-level experts within Alignment, and most others are coming up from within the field, consider allocating a significant budget (in terms of limited attendee slots) to inviting relatively senior people from other disciplines. Worst case, they are lost or bored or contribute useless ideas, and one slot was wasted for a few days. But if they engage well, you have a lot of information about their potential to contribute as a member of a team (hire them, collaborate, invite them to more things), and they have a lot of information about how exciting and important that might be.
Here’s the rub: people who are that senior and that good are busy and get a lot if invitations, and are selective about which to accept. So you may need to make a strong pitch explaining why alignment is an important problem and why you think their particular expertise would be valuable. The low hanging fruit is therefore to invite ones who have already expressed an interest, however tentatively. Invite pretty much all of those.
Disclosure 1: these observations are supported by my reading of history of science, my own experience switching fields, and experience in a leadership role promoting interdisciplinary collaboration in another fledgling research area.
Disclosure 2: these comments are potentially self-serving, speaking as a relatively senior member of an outside discipline who is interested in engaging with the alignment community, but not finding opportunities to engage at this sort of level, despite the widely professed value placed on diverse perspectives.
1. ^
  By the same token, the outsider brings with them a lot of other background knowledge, in quantity proportional to their maturity in the previous field, which is correct in proportion of the rigor of the previous field, and non-overlapping in proportion to the distance of the previous field.
What links here?
- foodforthought's comment on Trying to understand my own cognitive edge by Wei Dai (9 Nov 2025 16:01 UTC; 1 point)

foodforthought 1 Sep 2025 3:51 UTC
1 point
0
on: Why Artists Study Anatomy
Great post. I would have liked to see the images in this post but the links all appear to be broken. If the OP is here could you repair the links?
Based on the text alone, this strikes me as right on the mark.
An interesting bit of history: the New York Academy (which still exists, in another form) was back in the 1980s an unaccredited graduate school and the premiere training ground for classical figurative drawing and sculpture, which were otherwise in much neglect in the Art World. From what I have heard (second-hand), there were two competing schools within the Academy at the time, one group favoring “perceptual” drawing (essentially the skill of copying a 2D image, or seeing a model as a 2D image and then drawing what you literally see); and the other favoring “conceptual” drawing, the skill of understanding how objects in three-dimensional world generate the two-dimensional projection we see, and then drawing from an understanding of that underlying cause. I think the perceptual approach is typical of photo-realist painters (and most present day portrait artists), and the conceptual approach was typical of Renaissance painters.
An anecdote I love that illustrates the contrast is: apparently one day when the class was drawing a long pose the model took a break, and when she came back the pose was slightly different such that all the shadows changed. The Perceptual students complained, whereas a Conceptual student countered: actually we should change the lights every 15 minutes. Then we can see what is actually there, and draw it from a better understanding.
For an example of what drawing looks like when approached conceptually, see the drawings of Luca Cambiaso (1527-1585). (This is not an artwork; it is a conceptual study done to figure out the scene 3-dimensionally in preparation for a classical renaissance drawing or painting).

foodforthought 27 Jul 2025 19:50 UTC
3 points
0
on: HRT in Menopause: A candidate for a case study of epistemology in epidemiology, statistics & medicine
in response to my remark
What has particularly struck me in reading on this topic is the degree to which statements of fact or logic are often interwoven with rhetorical style that serves to sway opinion by means other than reason [...]
@the gears to ascension wrote
There’s recently been vigorous debate on lesswrong about whether that’s the case.
I am not sure what “that” refers to in “whether that’s the case” . Based on the segment of my post they quoted, it sounds like there’s been debate about whether it’s the case that facts and logic are often interwoven with rhetorical persuasion devices when making arguments. But I suspect the commenter is referring to my wondering whether it would be advantageous to avoid mixing facts and logic with rhetorical persuasion devices when making arguments. If there has been vigorous debate on lesswrong on the latter topic, I would like to read it. Can anyone provide links or keywords?
My interpretation of the commenter’s intent is supported by their further saying
the quoted claim is not clearly well enough justified to conclude the case about whether moderating impulses towards confident wording is useful.
I will note that I didn’t make a claim about this, much less attempt to provide any sort of evidence. So far I only suggested some possible arguments for and against the first thought that came to my mind. My intention was to invite a discussion on the pros and cons of this idea.

foodforthought 29 Jun 2025 23:48 UTC
11 points
0
in reply to: pmarc’s comment on: X explains Z% of the variance in Y
In my opinion, this post misses the main challenge with the “platonic ideal” vision of a concept like percentage of explained variance: it’s something that accidentally depends on the distribution of your variables in the sample, rather than a fundamental property of the relationship between those variables.

Perhaps we need to step back and clarify what “Platonic Explained Variance” could even mean. All knowledge is contextual; it is a mistake to expect that there is a Truth to be known devoid of context. I supposed that the OP meant by this phrase something like: the true, complete statistical dependence between X and Y in the sampled population, as against our estimate or approximation of that dependence based on a given limited sample or statistical model. In any case, I’d like to argue that such distinction makes sense, while it does not make sense to look for a statistical relationship between X and Y that is eternally and universally true, independent of a specific population.

When we are using empirical statistics to describe the relationship between measurable variables X and Y, I think the conclusions we draw are always limited to the population we sampled. That is the essential nature of the inference. Generalization to the sampled population as a whole carries some uncertainty, which we can quantify based on the size of the sample we used and the amount of variability we observed, subject to some assumptions (e.g., about the underlying distributions, or independence of our observations).

But generalization to any other population always entails additional assumptions. If the original sample was limited in scope (e.g. a particular age, sex, geographic location, time point, or subculture), generalization outside that scope entails a new conjecture that the new group is essentially the same as the original one in every respect relevant to the claim. To the extent the original sample was broad in scope, we can as you say test whether such other factors detectably modified the association between X and Y, and if so, include these effects as covariates in our statistical model. As you note, this requires a lot of statistical power. Even so, whenever we generalize outside that population, we assume the new population is similar in the ways that matter, for both the main association and the modifier effects.

A statistical association can be factually, reproducibly true of a population and still be purely accidental, in which case we don’t expect it to generalize. When we generalize to a new context or group or point in time, I think we are usually relying on an (implicit or explicit) model that the observed statistical relation between X and Y is a consequence of underlying causal mechanisms. If and to the extent that we know what causal mechanisms are at play, we have a basis for predicting or checking whether the relevant conditions apply in any new context. But (1) generalization of the causal mechanism to a new condition is still subject to verification; a causal model derived in a narrow context could be incomplete, and the new condition may differ in a way that turns out to be causally important in a way we didn’t suspect; and (2) even if the causal mechanism perfectly generalizes, we do not expect “the fraction of variance explained” to generalize universally. That value depends on a plethora of other random and causal factors that will in general be different between populations [^1].

Summing up, I think it’s a mistake to look for the ‘Platonic Variance Explained’ divorced from a specific population. But we can meaningfully ask if the statistical dependence we estimated from a finite empirical sample using a particular statistical model accurately reflects the true and complete statistical dependence between the variables in the population from which we sampled.
1. This account might be particular to the branches of natural science that seek mechanistic causal models and/or fundamental theories as explanations. Other fields of research or philosophic frameworks that lack or eschew causal explanation or theory may have a different epistemic account, which I’d be interested to hear about.