I was indeed assuming that the model is not open-source and that control of the the inference infrastructure is good. So bad actors inside the company, or able to strong-arm the company or penetrate ist security are still a problem, As indeed are bad actors who can develop their own model.
Since my proposed means of control is token based, the required controls could simply be hardwired into the tokenizer code used for inference, so that abuse of the model would require sufficient access to change the tokenizer code. That in turn could be monitored using stabdard software security approaches (access controls, signing, and so forth).
If the model is actually superintelligent, then it generally simulating misaligned humans as being less smart than it is seems likely to be the default behavior, since humans are less smart than it. However, this can probably be bent a little by prompting (John Von Neuman was human, after all), and simulating a large number of misaligned John Von Neuman equivalents all cooperating at high speed (a nation of criminal masterminds in a data center) is not entirely safe, even if we have an even smarter superintelligent aligned AI to try to ride herd/do law enforcement on them.
Simulating anything unaligned as less capable rather remains me of the old crime shows of the 1950s and 1960s where it seemed to be an unwritten rule that the show writers were operating under that all criminals must be stupid and very bad planning skills. Nevertheless, it does seem like a good idea from an alignment point of view, if we could figure out how to implement it. (Fortunately for now all AI has bad planning skills, but that clearly isn’t going to last.)
I was indeed assuming that the model is not open-source and that control of the the inference infrastructure is good. So bad actors inside the company, or able to strong-arm the company or penetrate ist security are still a problem, As indeed are bad actors who can develop their own model.
Since my proposed means of control is token based, the required controls could simply be hardwired into the tokenizer code used for inference, so that abuse of the model would require sufficient access to change the tokenizer code. That in turn could be monitored using stabdard software security approaches (access controls, signing, and so forth).
If the model is actually superintelligent, then it generally simulating misaligned humans as being less smart than it is seems likely to be the default behavior, since humans are less smart than it. However, this can probably be bent a little by prompting (John Von Neuman was human, after all), and simulating a large number of misaligned John Von Neuman equivalents all cooperating at high speed (a nation of criminal masterminds in a data center) is not entirely safe, even if we have an even smarter superintelligent aligned AI to try to ride herd/do law enforcement on them.
Simulating anything unaligned as less capable rather remains me of the old crime shows of the 1950s and 1960s where it seemed to be an unwritten rule that the show writers were operating under that all criminals must be stupid and very bad planning skills. Nevertheless, it does seem like a good idea from an alignment point of view, if we could figure out how to implement it. (Fortunately for now all AI has bad planning skills, but that clearly isn’t going to last.)