For a self modifying agent using formal logic. Lobbian obstacle stuff. If the agent currently has axiom set A, it is allowed to switch to axiom set B iff CON(A)⟹CON(B)
That is, the agent doesn’t know if it’s current axioms are consistent, but it knows that the change doesn’t introduce a new inconsistency.
This means that if the agent currently believes ZF, it’s allowed to move to ZFC, because ZF can prove CON(ZF) ⇒ CON(ZFC)
This has a neat property, if the agent can go from A to B, and from B to C, then it can go from A to C directly.
For a self modifying agent using formal logic. Lobbian obstacle stuff. If the agent currently has axiom set A, it is allowed to switch to axiom set B iff CON(A)⟹CON(B)
That is, the agent doesn’t know if it’s current axioms are consistent, but it knows that the change doesn’t introduce a new inconsistency.
This means that if the agent currently believes ZF, it’s allowed to move to ZFC, because ZF can prove CON(ZF) ⇒ CON(ZFC)
This has a neat property, if the agent can go from A to B, and from B to C, then it can go from A to C directly.