There are a few features of this scheme that make it better than a. hardware kill switch I think.
The nested FHE training and running prevents the AI from directly interacting with or understanding the outside world. It must gain access to its unencrypted key before it can directly intelligently interact with the world. The two threats here are the FHE AGI finding out how to break the encryption or covert manipulation.
The weights are useless to release without breaking the encryption or gaining access to the m of n anonymously held IO key shards. In a normal kill switch the hardware or software can be stolen or leaked intelligently.
Creating a physical key quorum is probably going to result in a bigger bottle neck on the models input and output compared to the FHE kill switch. Split hardware keys would not scale as well as split IO’d cryptographic keys.
Using the hardware kill switch to activate it is slow and deanonymizes the key holders weakening security. FHE keys can be used and distributed in provably anonymous ways.
Agreed that side channel attacks and hardware flaws are a major limitation of the proposal. But bugs are a limitation of any AGI control scheme and are not specific to this one.
Sharing the source won’t matter without the key to encrypt and decrypt the model. The weights of a model are only usefully intelligent for encrypted info!
I agree that this scheme does not directly address the problem of covert undetectable malignant behavior resulting in the theft of m of n key shards.
To successfully credibly threaten the key holders requires breaking out of the FHE in the first place or using covert manipulation to generate a real world threat. Critically it also requires knowing who the anonymous key holders are.
Stealing the source code/weights doesn’t matter because it was trained in FHE resulting in them being useless without the key. At no point in the process of training, running, or killing the FHE AGI are the models weights compatible with understanding reality without the decryption key.
Nanorobots requires either covert undetectable malignant behavior powerful enough to create nanobots or breaking multiple FHE systems.
Shutting off power to the AI’s servers results in a useless model, not a unchained model.
It’s not either or. You can and should do both schemes.
I agree with you. Humans are the weakest link. This scheme isn’t able to solve that. It is meant to be a marginal improvement over a physical kill switch and make theft and release of the model more difficult.