Now suppose that every point in space is Topologically Indistinguishable (as in our universe).
Then initializeUniverse() or universe.step() must somehow break the symmetry of the initial state, perhaps through nondeterminism. Simple universes that put a lot of weight on one timeline will be asymmetric, right?
much more likely than a machine with the property “accurately predicts my data and then do something malicious”
The idea is that “accurately predicts my data” is implied by “do something malicious”, which you will find contains one fewer word :P.
This means it is literally impossible for an agent inside a Turing Machine to out-think you.
In Robust Cooperation in the Prisoner’s Dilemma, agents each prove that the other will cooperate. The halting problem may be undecidable in the general case, but haltingness can sure be proven/disproven in many particular cases.
I don’t get why we are assuming the bridge rules will be complicated?
I don’t expect our own bridge rules to be simple: Maxwell’s equations look simple enough, but locating our Earth in the quantum multiverse requires more bits of randomness than there are atoms.
You choosing your actions is compatible with a deterministic universe. https://www.lesswrong.com/posts/NEeW7eSXThPz7o4Ne/thou-art-physics
Then
initializeUniverse()
oruniverse.step()
must somehow break the symmetry of the initial state, perhaps through nondeterminism. Simple universes that put a lot of weight on one timeline will be asymmetric, right?The idea is that “accurately predicts my data” is implied by “do something malicious”, which you will find contains one fewer word :P.
In Robust Cooperation in the Prisoner’s Dilemma, agents each prove that the other will cooperate. The halting problem may be undecidable in the general case, but haltingness can sure be proven/disproven in many particular cases.
I don’t expect our own bridge rules to be simple: Maxwell’s equations look simple enough, but locating our Earth in the quantum multiverse requires more bits of randomness than there are atoms.