I am not sure if labels help here. I’m simply pointing out that logical counterfactuals applied to the “real Lincoln” lead to the sort of issues MIRI is facing right now when trying to make progress in the theoretical AI alignment issues. The reference class approach removes the difficulties, but then it is hard to apply it to the “mathematical facts”, like what is the probability of 100...0th digit of pi being 0 or, to quote the OP “If the Modularity Theorem were false...” and the prevailing MIRI philosophy does not allow treating logical uncertainty as environmental.
Sure. In the case of Lincoln, I would say the problem is solved by models even as clean as Pearl-ian causal networks. But in math, there’s no principled causal network model of theorems to support counterfactual reasoning as causal calculus.
Of course, I more or less just think that we have an unprincipled causality-like view of math that we take when we think about mathematical counterfactuals, but it’s not clear that this is any help to MIRI understanding proof-based AI.
I am not sure if labels help here. I’m simply pointing out that logical counterfactuals applied to the “real Lincoln” lead to the sort of issues MIRI is facing right now when trying to make progress in the theoretical AI alignment issues. The reference class approach removes the difficulties, but then it is hard to apply it to the “mathematical facts”, like what is the probability of 100...0th digit of pi being 0 or, to quote the OP “If the Modularity Theorem were false...” and the prevailing MIRI philosophy does not allow treating logical uncertainty as environmental.
Sure. In the case of Lincoln, I would say the problem is solved by models even as clean as Pearl-ian causal networks. But in math, there’s no principled causal network model of theorems to support counterfactual reasoning as causal calculus.
Of course, I more or less just think that we have an unprincipled causality-like view of math that we take when we think about mathematical counterfactuals, but it’s not clear that this is any help to MIRI understanding proof-based AI.
I don’t think I am following your argument. I am not sure what Pearl’s causal networks are and how they help here, so maybe I need to read up on it.