Thanks Peter for the post and thank you everyone for the comments. Let me try to clarify a bit. We’re looking for an absolute foundation of trust on top of which we can build a world safe for AGI. We believe that we need to adopt a “Security Mindset” in which AGI’s either on their own or controlled by malicious humans need to be considered a full on adversaries. The only two absolute guarantees that we have are mathematical proof and the laws of physics. Even the most powerful AGI can’t prove a falsehood or violate the laws of physics. Based on these we show how to build a network of “provable contracts” that provide absolute guardrails around dangerous actions. As a commenter points out, figuring out which actions we need to protect against and what the rules should be is absolutely essential and not at all trivial! In fact, I believe that should be one of the primary activities of humanity for the next decade!
Thanks Peter for the post and thank you everyone for the comments. Let me try to clarify a bit. We’re looking for an absolute foundation of trust on top of which we can build a world safe for AGI. We believe that we need to adopt a “Security Mindset” in which AGI’s either on their own or controlled by malicious humans need to be considered a full on adversaries. The only two absolute guarantees that we have are mathematical proof and the laws of physics. Even the most powerful AGI can’t prove a falsehood or violate the laws of physics. Based on these we show how to build a network of “provable contracts” that provide absolute guardrails around dangerous actions. As a commenter points out, figuring out which actions we need to protect against and what the rules should be is absolutely essential and not at all trivial! In fact, I believe that should be one of the primary activities of humanity for the next decade!