Frederik Hytting Jørgensen

• Frederik Hytting Jørgensen 14 Aug 2025 10:04 UTC

  2 points

  2

  on: How Does A Blind Model See The Earth?

  Fun project.

  I think these kinds of pictures ‘underestimate’ models’ geographical knowledge. Just imagine having a human perform this task. The human may have very detailed geographical knowledge, may even be able to draw a map of the world from memory. This does not imply that they would be able to answer questions about latitude and longitude.

• Frederik Hytting Jørgensen 7 Jul 2025 9:48 UTC

  2 points

  0

  in reply to: Alex Gibson’s comment on: Small foun­da­tional puz­zle for causal the­o­ries of mechanis­tic interpretability

  Am I right that the line of argument here is not about the generalization properties, but a claim about the quality of explanation, even on the restricted distribution?

  Yes, I think that is a good way to put it. But faithful mechanistic explanations are closely related to generalization.

  Like here, your causal model $M^{\ast }$ should have the explicit condition “x_1=x_2”.

  That would be a sufficient condition for $M^{\ast }$ to make the correct predictions. But that does not mean that $M^{\ast }$ provides a good mechanistic explanation of $M$ on those inputs.

• Frederik Hytting Jørgensen 6 Jul 2025 12:45 UTC

  2 points

  0

  in reply to: ParrotRobot’s comment on: Small foun­da­tional puz­zle for causal the­o­ries of mechanis­tic interpretability

  I’m a bit unsure about the way you formalize things, but I think I agree with your point. It is a helpful point. I’ll try to state a similar (same?) point.

  Assume that all variables have the natural numbers as their domain. Assume WLOG that all models only have one input and one output node. Assume that $M^{\ast }$ is an abstraction of $M$ on relative to input support $I=\left[n\right]$ and $\tau$. Now there exists a model $M^{+}$ such that $M\left(j\right)=M^{+}\left(j\right)$ for all $j\in I$, but $M^{\ast }$ is not a valid abstraction of $M^{+}$relative to input support $I^{+}=[n+1]$. For example, you may define the structural assignment of the output node in $M^{+}$ by

  $$F_{\text{output}}^{M^{+}}\left(X^{+}\right):=\{\begin{array}{cc}x& X_{\text{input}}^{+}\ge n+1\\ F_{\text{output}}^M\left(X^{+}\right)& X_{\text{input}}^{+}\in \left[n\right]\end{array},$$

  where $x$ is an element in $N\setminus {\tau }_{\text{output}}^{-1}\left(M^{\ast }\right({\tau }_{\text{input}}(n+1)\left)\right[\text{output}\left]\right)$, which we assume to be non-empty.

  There is nothing surprising about this. As you say, we need assumptions to rule things like these out. And coming up with those assumptions seems potentially interesting. People working on mechanistic interpretability should think more about what assumptions would make their methods reasonable.

  The main point of the post is not that causal abstractions do not provide guarantees about generalization (this point is underappreciated, but really, why would they?). My main point is that causal abstractions can misrepresent the mechanistic nature of the underlying model (this is of course related to generalizability).

Small foun­da­tional puz­zle for causal the­o­ries of mechanis­tic interpretability

• Frederik Hytting Jørgensen 3 Feb 2023 13:34 UTC

  1 point

  0

  in reply to: Magdalena Wache’s comment on: Finite Fac­tored Sets in Pictures

  Finally got around to looking at this. I didn’t read the paper carefully, so I may have missed something, but I could not find anything that makes me more at ease with this conclusion.

  Ben has already shown that it is perfectly possible that Y causes X. If this is somehow less likely that X causes Y, this is exactly what needs to be made precise. If faithfulness is the assumption that makes this work, then we need to show that faithfulness is a reasonable assumption in this example. It seems that this work has not been done?

  If we can find the precise and reasonable assumptions that exclude that Y causes X, that would be super interesting.

• Frederik Hytting Jørgensen 16 Dec 2022 16:47 UTC

  2 points

  0

  in reply to: Magdalena Wache’s comment on: Finite Fac­tored Sets in Pictures

  For example, in theorem 3.2 in Causation, Prediction, and Search, we have a result that says that faithfulness holds with probability 1 if we have a linear model with coefficients drawn randomly from distributions with positive densities.
  
  It is not clear to me why we should expect faithfulness to hold in a situation like this, where Z is constructed from other variables with a particular purpose in mind.

  Consider the graph Y<-X->Z. If I set Y:=X and Z:=X, we have that X⊥Y|Z, violating faithfulness. How are you sure that you don’t violate faithfulness by constructing Z?

• Frederik Hytting Jørgensen 15 Dec 2022 15:59 UTC

  1 point

  0

  in reply to: Magdalena Wache’s comment on: Finite Fac­tored Sets in Pictures

  I’m not quite convinced by this response. Would it be possible to formalize “set of probability distributions in which Y causes X is a null set, i.e. it has measure zero.”?

  It is true that if the graph was (Y->X, X->Z, Y->Z), then we would violate faithfulness. There are results that show that under some assumptions, faithfulness is only violated with probability 0. But those assumptions do not seem to hold in this example.