foodforthought

• foodforthought 1 Sep 2025 3:51 UTC

  1 point

  0

  on: Why Artists Study Anatomy

  Great post. I would have liked to see the images in this post but the links all appear to be broken. If the OP is here could you repair the links?

  Based on the text alone, this strikes me as right on the mark.

  An interesting bit of history: the New York Academy (which still exists, in another form) was back in the 1980s an unaccredited graduate school and the premiere training ground for classical figurative drawing and sculpture, which were otherwise in much neglect in the Art World. From what I have heard (second-hand), there were two competing schools within the Academy at the time, one group favoring “perceptual” drawing (essentially the skill of copying a 2D image, or seeing a model as a 2D image and then drawing what you literally see); and the other favoring “conceptual” drawing, the skill of understanding how objects in three-dimensional world generate the two-dimensional projection we see, and then drawing from an understanding of that underlying cause. I think the perceptual approach is typical of photo-realist painters (and most present day portrait artists), and the conceptual approach was typical of Renaissance painters.

  An anecdote I love that illustrates the contrast is: apparently one day when the class was drawing a long pose the model took a break, and when she came back the pose was slightly different such that all the shadows changed. The Perceptual students complained, whereas a Conceptual student countered: actually we should change the lights every 15 minutes. Then we can see what is actually there, and draw it from a better understanding.

  For an example of what drawing looks like when approached conceptually, see the drawings of Luca Cambiaso (1527-1585):
  

  

• foodforthought 27 Jul 2025 19:50 UTC

  3 points

  0

  on: HRT in Menopause: A can­di­date for a case study of episte­mol­ogy in epi­demiol­ogy, statis­tics & medicine

  in response to my remark

  What has particularly struck me in reading on this topic is the degree to which statements of fact or logic are often interwoven with rhetorical style that serves to sway opinion by means other than reason [...]

  @the gears to ascension wrote

  There’s recently been vigorous debate on lesswrong about whether that’s the case.

  I am not sure what “that” refers to in “whether that’s the case” . Based on the segment of my post they quoted, it sounds like there’s been debate about whether it’s the case that facts and logic are often interwoven with rhetorical persuasion devices when making arguments. But I suspect the commenter is referring to my wondering whether it would be advantageous to avoid mixing facts and logic with rhetorical persuasion devices when making arguments. If there has been vigorous debate on lesswrong on the latter topic, I would like to read it. Can anyone provide links or keywords?

  My interpretation of the commenter’s intent is supported by their further saying

  the quoted claim is not clearly well enough justified to conclude the case about whether moderating impulses towards confident wording is useful.

  I will note that I didn’t make a claim about this, much less attempt to provide any sort of evidence. So far I only suggested some possible arguments for and against the first thought that came to my mind. My intention was to invite a discussion on the pros and cons of this idea.

HRT in Menopause: A can­di­date for a case study of episte­mol­ogy in epi­demiol­ogy, statis­tics & medicine

• foodforthought 29 Jun 2025 23:48 UTC

  11 points

  0

  in reply to: pmarc’s comment on: X ex­plains Z% of the var­i­ance in Y

  In my opinion, this post misses the main challenge with the “platonic ideal” vision of a concept like percentage of explained variance: it’s something that accidentally depends on the distribution of your variables in the sample, rather than a fundamental property of the relationship between those variables.

  Perhaps we need to step back and clarify what “Platonic Explained Variance” could even mean. All knowledge is contextual; it is a mistake to expect that there is a Truth to be known devoid of context. I supposed that the OP meant by this phrase something like: the true, complete statistical dependence between X and Y in the sampled population, as against our estimate or approximation of that dependence based on a given limited sample or statistical model. In any case, I’d like to argue that such distinction makes sense, while it does not make sense to look for a statistical relationship between X and Y that is eternally and universally true, independent of a specific population.

  When we are using empirical statistics to describe the relationship between measurable variables X and Y, I think the conclusions we draw are always limited to the population we sampled. That is the essential nature of the inference. Generalization to the sampled population as a whole carries some uncertainty, which we can quantify based on the size of the sample we used and the amount of variability we observed, subject to some assumptions (e.g., about the underlying distributions, or independence of our observations).

  But generalization to any other population always entails additional assumptions. If the original sample was limited in scope (e.g. a particular age, sex, geographic location, time point, or subculture), generalization outside that scope entails a new conjecture that the new group is essentially the same as the original one in every respect relevant to the claim. To the extent the original sample was broad in scope, we can as you say test whether such other factors detectably modified the association between X and Y, and if so, include these effects as covariates in our statistical model. As you note, this requires a lot of statistical power. Even so, whenever we generalize outside that population, we assume the new population is similar in the ways that matter, for both the main association and the modifier effects.

  A statistical association can be factually, reproducibly true of a population and still be purely accidental, in which case we don’t expect it to generalize. When we generalize to a new context or group or point in time, I think we are usually relying on an (implicit or explicit) model that the observed statistical relation between X and Y is a consequence of underlying causal mechanisms. If and to the extent that we know what causal mechanisms are at play, we have a basis for predicting or checking whether the relevant conditions apply in any new context. But (1) generalization of the causal mechanism to a new condition is still subject to verification; a causal model derived in a narrow context could be incomplete, and the new condition may differ in a way that turns out to be causally important in a way we didn’t suspect; and (2) even if the causal mechanism perfectly generalizes, we do not expect “the fraction of variance explained” to generalize universally. That value depends on a plethora of other random and causal factors that will in general be different between populations [^1].

  Summing up, I think it’s a mistake to look for the ‘Platonic Variance Explained’ divorced from a specific population. But we can meaningfully ask if the statistical dependence we estimated from a finite empirical sample using a particular statistical model accurately reflects the true and complete statistical dependence between the variables in the population from which we sampled.

  1. This account might be particular to the branches of natural science that seek mechanistic causal models and/​or fundamental theories as explanations. Other fields of research or philosophic frameworks that lack or eschew causal explanation or theory may have a different epistemic account, which I’d be interested to hear about.

• foodforthought 22 Jun 2025 0:21 UTC

  6 points

  0

  in reply to: Leon Lang’s comment on: X ex­plains Z% of the var­i­ance in Y

  Note that parts of my post are actually model-free! For example, the mathematical definition and the example of twin studies do not make use of a model

  Yes, good point, I should have said “unlike regression” rather than “unlike variance explained”. I’ll have to think more on how the type of analysis described in the twin example maps onto information theory.

  But this is predicated on the implicit model that Y is a normally distributed variable.

  I’m not aware of (implicitly) making that assumption in my post!

  By “this” I meant the immediately preceding statements in my post. (Although the cartoon distributions you show do look normal-ish, so at least you invite that intuition). The idea that the mean or average is a good measure of central tendency of a distribution, or a good estimator, is so familiar we forget that it requires justification. For Normal distributions, it is the lowest MSE estimator, the maximum likelihood estimator, and is an unbiased estimator, but this isn’t true of all distributions. For a skewed, long-tailed distribution, for example, the median is a better estimator. For a binomial distribution, the mean is almost never the maximum likelihood estimator. For a Cauchy distribution, the mean is not even defined (although to be fair I’m not entirely sure entropy is well defined in that case, either). Likewise the properties of variance that make it a good estimator of dispersion for a Normal distribution don’t necessarily make it good for other distributions.

  It is true that partitioning of variance and “variance explained” as such don’t rely on a normality assumption, and there are non-parametric versions of regression, correlation, ANOVA etc. that don’t assume normality. So I have not entirely put my finger on what the difference is.

  You can measure mutual information even if the form of the relationship is unknown or complicated.

  Is this so? Suppose we’d want to measure differential entropy, as a simplified example, and the true density “oscillates” a lot. In that case, I’d expect that the entropy is different than what it is if the density were smoother. But it might be hard to see the difference in a small dataset. The type of regularity/​simplicity assumptions about the density might thus influence the result.

  This might be a good place to mention that I work exclusively with discrete entropy, and am not very familiar with notations or proofs in differential (continuous) entropy. So if Y is continuous, in practice this involves discretizing the value of Y (binning your histograms). I agree the continuous case would be more directly comparable, but I don’t think this is likely to be fundamentally important, do you?

  In principle, conceptually, you can estimate entropy directly from the probability density function (PDF) non-parametrically as H = sum(-P log2 P), where the sum is over all possible values of Y, and P is the probability Y takes on a given value.^[1] Likewise, you can estimate the mutual information directly from the joint probability distribution between X and Y, the equation for which I won’t try to write out here without an equation editor. In practice, if Y is continuous, the more data you have, the more finely you can discretize Y and the more subtly you can describe the shape of the distribution, so you converge on the true PDF and thus the true entropy as the data size goes to infinity.

  I’m not denying that it can take a lot of data to measure entropy or mutual information by brute force in this way. What is worse, these naive estimators are biased if distributions are under-sampled. So getting a good estimate of entropy or mutual information from data is very tricky, and the shape of the distribution can make the estimation more or less tricky. To the extent one relies on regularity or simplicity assumptions to overcome data limitations, these assumptions can affect your result.

  Still, if you are careful about it, an estimate based on assumptions can still be a strict bound: X removes at least z% of your uncertainty about Y. There is a direct analogy in regression models: if Yhat=f(X) explains z% of the variance of y (assuming this is established properly), then x “Platonically” explains at least z% of the variance of y.

  Relatedly, you can pre-process X into some derived variable such as Q=f(X) or an estimator Yhat=f(X), and then measure the mutual information between the derived variable and the true value of Y. The Data Processing Inequality states that if the derived variable contains Z amount of information about Y, the input variable X must contain at least that much information. This is very much like defining a particular regression model f(X); and in the Yhat=f(X) case, it does give you a model you can use to predict Y from X.

  1. ^

     sorry, I haven’t figured out the equation editor yet.

• foodforthought 21 Jun 2025 18:09 UTC

  26 points

  0

  on: X ex­plains Z% of the var­i­ance in Y

  TL;DR: Information Theory is another mathematical framework for understanding the relatedness of X and Y based on measured joint probability distributions.

  Epistemic status—quite confident. I would not be qualified to teach a course on this for graduate students in mathematics or statistics, but I do teach a course on this for PhD students in biology and psychology.

  Well done! In the spirit of “yes, and”, I think it is worth pointing out that there are highly related concepts in Shannon Information Theory, and wonder if this comes closer to the OP’s notion of Platonic Variance Explained. (If there’s interest in a longer explanation of Info. Theory with equations and graphs and examples, I could write that as a post; but perhaps that’s already well-tread ground here. For now I just want to make a quick comparison, without fully fleshing out the points).

  In the “explained variance” framework we think about the variability of possible Y’s in terms of its spread around (literally, distance from) the mean value of Y. If you don’t know what Y is in a particular case and you are forced to guess, often the average Y is a good guess in two senses: first, on average this guess has the lowest error (squared difference between the guess and the actual value of Y); and second, it is more frequently correct than any other choice. But this is predicated on the implicit model that Y is a normally distributed variable. Due to the Central Limit Theorem, many variables are approximately normally distributed, so this model is often a useful one. Regression then builds on this to quantify the extent to which X constrains the possible value of Y, as nicely explained above. The OP nicely explains that “variance explained” means the extent to which some particular model of Y based on X (such as: Y=mX+b), provides a better guess than the alternative model of Y=the average Y in the first sense mentioned above. This statement is at least as much a statement about the model, as about X.

  Information Theory measures the same things from the same data, through a different conceptual lens.

  Instead of measuring the variability of Y by its variance, Information Theory measures this by its entropy, which you can think of is the uncertainty you have about Y before you observe it. Like the variance, the entropy only depends on the distribution of Y. But unlike variance, entropy makes no assumptions about the shape of the distribution of Y. The values of Y can have an arbitrarily complex, non-parametric distribution, for which the mean value of Y might be an extremely bad guess in both senses mentioned above.

  Instead of measuring the relationship between X and Y by the variance explained, Information Theory measures the mutual information between X and Y, which you can think of as how much your uncertainty about Y is reduced by knowing X. Like “variance explained”, mutual information depends only on the joint distribution of X and Y. But unlike regression, it does not require positing any model of how Y depends on X. This is powerful, because it gives you a fact about the X-Y relationship, not a fact about the goodness of some model. You can measure mutual information even if the form of the relationship is unknown or complicated. On the down side, since it’s model-free, it doesn’t directly give you a method for predicting Y from X.

  I think Mutual Information comes closer to what the OP calls the Platonic Variance Explained, but correct me if I’m wrong!

  I’m restricting myself to the main theme of the OP here, but it might be interesting flesh out other ways the two frameworks are similar or different, such as issues of under-sampling/​overfitting, independence-of-samples assumption; correlation vs causation; data processing inequality...