That’s a great question, thanks for thinking about the implications!
Both unsupervised learning and nonparametric statistics still require you to define an input representation and impose structural constraints (independence, a kernel, a distance metric) that the discovered variables must satisfy — so the subjectivity isn’t eliminated, just relocated to the algorithm and representation level. They’re restricting the set of possible models to those satisfying independence, etc.
But how would you know if those are tenable assumptions? By actually looking at what they produce on examples where you already know in advance “what looks right.” That’s the whole point: the assumption-selection problem is prior to any statistical methodology, whether parametric or not, supervised or not. No amount of mathematical gerrymandering can reduce the burden of subjectivity.
Not to mention, these methods are generally fit to observational data, and the set of interventional distributions uniquely recoverable from observational data alone is measure zero in the space of all compatible causal models (see “On Pearl’s Hierarchy and the Foundations of Causal Inference” by Bareinboim et al., 2022). If you try to account for this by imposing causal structure via further assumptions, you’re starting to do causal representation learning, which I mention in footnote 1.
I don’t think this damns science generally (nor mech interp specifically)… so long as we share the “subjective” parts of our modeling and not just the “objective math+plots.” Acknowledging and openly discussing how to allocate our unavoidable subjectivity-budget is the only way to avoid talking past each other. We can’t address blind spots we don’t let others see.
That’s a great question, thanks for thinking about the implications!
Both unsupervised learning and nonparametric statistics still require you to define an input representation and impose structural constraints (independence, a kernel, a distance metric) that the discovered variables must satisfy — so the subjectivity isn’t eliminated, just relocated to the algorithm and representation level. They’re restricting the set of possible models to those satisfying independence, etc.
But how would you know if those are tenable assumptions? By actually looking at what they produce on examples where you already know in advance “what looks right.” That’s the whole point: the assumption-selection problem is prior to any statistical methodology, whether parametric or not, supervised or not. No amount of mathematical gerrymandering can reduce the burden of subjectivity.
Not to mention, these methods are generally fit to observational data, and the set of interventional distributions uniquely recoverable from observational data alone is measure zero in the space of all compatible causal models (see “On Pearl’s Hierarchy and the Foundations of Causal Inference” by Bareinboim et al., 2022). If you try to account for this by imposing causal structure via further assumptions, you’re starting to do causal representation learning, which I mention in footnote 1.
I don’t think this damns science generally (nor mech interp specifically)… so long as we share the “subjective” parts of our modeling and not just the “objective math+plots.” Acknowledging and openly discussing how to allocate our unavoidable subjectivity-budget is the only way to avoid talking past each other. We can’t address blind spots we don’t let others see.