How is the agent supposed to know which parts of the world are “opponents” that it should make unpredictable? This seems like a potentially show-stopping problem with your approach and I’m surprised it’s not listed under open problems. Is there something obvious that I’m missing?
This post doesn’t propose a complete decision algorithm, it merely collects several applications of what appears to be the same trick. Your question seems to fall in the scope of the third open problem: perhaps it’s a good idea to make all sorts of facts (events in the world) unconditionally unpredictable (unpredictable when not assuming what agent’s own decision would be), if at the same time it’s possible to retain their conditional predictability (predictability when an assumption about agent’s own decision is made).
(Making an event unconditionally unpredictable but conditionally predictable looks like a good candidate for what gaining control over that event means, and an agent would want to gain control over as many events as possible. In particular, gaining control over your opponent’s decisions is probably a good idea.)
How is the agent supposed to know which parts of the world are “opponents” that it should make unpredictable? This seems like a potentially show-stopping problem with your approach and I’m surprised it’s not listed under open problems. Is there something obvious that I’m missing?
This post doesn’t propose a complete decision algorithm, it merely collects several applications of what appears to be the same trick. Your question seems to fall in the scope of the third open problem: perhaps it’s a good idea to make all sorts of facts (events in the world) unconditionally unpredictable (unpredictable when not assuming what agent’s own decision would be), if at the same time it’s possible to retain their conditional predictability (predictability when an assumption about agent’s own decision is made).
(Making an event unconditionally unpredictable but conditionally predictable looks like a good candidate for what gaining control over that event means, and an agent would want to gain control over as many events as possible. In particular, gaining control over your opponent’s decisions is probably a good idea.)
(It might be good to keep in mind Eliezer’s point about how discorrelation is potentially a scarce resource (because correlation is a ratchet).)
That depends on the opponent’s computational power. There are certainly cryptographic approaches to this problem.