One natural approach is to try to specify an algorithm that uses reflective oracles to reason about itself under logical uncertainty (under axioms characterizing the reflective oracle), and then to implement UDT by conditioning on logical facts. I don’t know if this adds anything beyond implementing logical uncertainty with a halting oracle. If this doesn’t work there are a few other similar approaches (e.g. restrict to Sigma_1 sentences, or see if the oracle’s behavior can motivate the definition of an idealized math intuition module with nicer behavior than our current candidates).
I suspect that something like this could give us clearer insight into whether or not conditioning on logical facts is an acceptable method of taking counterfactuals. Right now that seems hard to settle, because the answer seems dependent on the algorithm used for reasoning under logical uncertainty. I think most people aren’t very optimistic about conditioning, but it’s still my leading contender (unless I’ve missed some recent discouraging news).
One natural approach is to try to specify an algorithm that uses reflective oracles to reason about itself under logical uncertainty (under axioms characterizing the reflective oracle), and then to implement UDT by conditioning on logical facts. I don’t know if this adds anything beyond implementing logical uncertainty with a halting oracle. If this doesn’t work there are a few other similar approaches (e.g. restrict to Sigma_1 sentences, or see if the oracle’s behavior can motivate the definition of an idealized math intuition module with nicer behavior than our current candidates).
I suspect that something like this could give us clearer insight into whether or not conditioning on logical facts is an acceptable method of taking counterfactuals. Right now that seems hard to settle, because the answer seems dependent on the algorithm used for reasoning under logical uncertainty. I think most people aren’t very optimistic about conditioning, but it’s still my leading contender (unless I’ve missed some recent discouraging news).