# Nisan

Karma: 3,912

NewTop

Yeah, I think the fact that Elo only models the macrostate makes this an imperfect analogy. I think a better analogy would involve a hybrid model, which assigns a probability to a chess game based on whether each move is plausible (using a policy network), and whether the higher-rated player won.

I don’t think the distinction between near-exact and nonexact models is essential here. I bet we could introduce extra entropy into the short-term gas model and the rollout would still be superior for predicting the microstate than the Boltzmann distribution.

Sure: If we can predict the next move in the chess game, we can predict the next move, then the

*next*, then the*next*. By iterating, we can predict the whole game. If we have a probability for each next move, we multiply them to get the probability of the game.Conversely, if we have a probability for an entire game, then we can get a probability for just the next move by adding up all the probabilities of all games that can follow from that move.

Thanks, I didn’t know that about the partition function.

In the post I was thinking about a situation where we know the microstate to some precision, so the simulation is accurate. I realize this isn’t realistic.

The sum isn’t over , though, it’s over all possible tuples of length . Any ideas for how to make that more clear?

# Beliefs at different timescales

I’m having trouble following this step of the proof of Theorem 4: “Obviously, the first conditional probability is 1”. Since the COD isn’t necessarily reflective, couldn’t the conditional be anything?

# Counterfactuals and reflective oracles

The linchpin discovery is probably February 2016.

Ok. I think that’s the way I should have written it, then.

The definition involving the permutation is a generalization of the example earlier in the post: is the identity and swaps heads and tails. And . In general, if you observe and , then the counterfactual statement is that if you had observed , then you would have also observed .

I just learned about probability kernels thanks to user Diffractor. I might be using them wrong.

Oh, interesting. Would your interpretation be different if the guess occurred well after the coinflip (but before we get to see the coinflip)?

That sounds about right to me. I think people have taken stabs at looking for causality-like structure in logic, but they haven’t found anything useful.

# Counterfactuals, thick and thin

What predictions can we get out of this model? If humans use counterfactual reasoning to initialize MCMC, does that imply that humans’ implicit world models don’t match their explicit counterfactual reasoning?

I agree exploration is a hack. I think exploration vs. other sources of non-dogmatism is orthogonal to the question of counterfactuals, so I’m happy to rely on exploration for now.

“Programmatically Interpretable Reinforcement Learning” (Verma et al.) seems related. It would be great to see modular, understandable glosses of neural networks.

# An environment for studying counterfactuals

I’d like to rescue/clarify Mitchell’s summary. The paper’s resolution of the Fermi paradox boils down to “(1) Some factors in the Drake equation are highly uncertain, and we don’t see any aliens, so (2) one or more of those factors must be small after all”.

(1) is enough to weaken the argument for aliens, to the point where there’s no paradox anymore. (2) is basically Section 5 from the paper (“Updating the factors”).

The point you raised, that “expected number of aliens is high vs. substantial probability of no aliens” is an explanation of why people were confused.

I’m making this comment because if I’m right it means that we only need to look for people (like me?) who were saying all along “there is no Fermi paradox because abiogenesis is cosmically rare”, and figure out why no one listened to them.

I heard a similar story about when Paul Sally visited a grade school classroom. He asked the students what they were learning, and they said “Adding fractions. It’s really hard, you have to find the greatest common denominator....” Sally said “Forget about that, just multiply the numerator of each fraction by the denominator of the other and add them, and that’s your numerator.” The students loved this, and called it the Sally method.

Thanks, I made this change to the post.