All the visions I have been considering have the AI starting at a public cloud like Amazon’s EC2 or some private data center like Google’s. While I am sure there is some slack that an AI could take advantage of, I can’t imagine there is enough slack to do the computations needed for the scheming and duplicity.
If on the other hand, when human and beyond level AI is realized and we find that most personal compute devices are fully capable to perform at that level—then I could see the chance for the scheming AI. Is that a common vision for how this might play out? Are there others?
I don’t think there’s one specific common vision about how a scheming AI would emerge.
I’m just extrapolating from my experience as someone who programs computers but can’t even pinpoint why my e-book reader freezes up when it does. Twenty-first century computers are already so complex that no person can track what they do in real-time, and as computers and their programs become more complex, tracking their behaviour is going to become an even harder task. (The obvious way out is to implement meta-programs and/or meta-computers which do that job, but then of course we have to ask how we can keep a close eye on those.)
Also in this future, the monitoring software the AI’s owner might use would also be near AI level intelligent.
A set of specialized oracles could be used to monitor inputs, internal computations and outputs. One oracle keeps records of every input and output. The question to this oracle is always the same: Is the AI lying? Another oracle is tasked with input steam analysis to filter out any taboo chunks. Other oracles can serve to monitor internal thought processes and self-improvement steps.
If these safeguarding oracles are strictly limited in their capabilities they do not pose a risk to the outside world. The core of such oracles could consist of straight forward defined heuristical rule sets.
Any of above mentioned oracles can be validated and independently certified. This would create a framework for tools, genies and sovereigns with assured friendliness.
Two problems: First, while you have an intuition that something is “scheming” the AI may not have it- to the AI it may be simply attempting resouce optimization. Second, putting things in a cloud is a really bad idea- that would give the AI broad access and if it doesn’t have very good safeguards it could very likely get access to surrounding resources or information in the cloud it isn’t supposed to have.
Doubtful. I can’t even keep track of all of the computations done by the computer I’m using to write this comment.
All the visions I have been considering have the AI starting at a public cloud like Amazon’s EC2 or some private data center like Google’s. While I am sure there is some slack that an AI could take advantage of, I can’t imagine there is enough slack to do the computations needed for the scheming and duplicity.
If on the other hand, when human and beyond level AI is realized and we find that most personal compute devices are fully capable to perform at that level—then I could see the chance for the scheming AI. Is that a common vision for how this might play out? Are there others?
I don’t think there’s one specific common vision about how a scheming AI would emerge.
I’m just extrapolating from my experience as someone who programs computers but can’t even pinpoint why my e-book reader freezes up when it does. Twenty-first century computers are already so complex that no person can track what they do in real-time, and as computers and their programs become more complex, tracking their behaviour is going to become an even harder task. (The obvious way out is to implement meta-programs and/or meta-computers which do that job, but then of course we have to ask how we can keep a close eye on those.)
A set of specialized oracles could be used to monitor inputs, internal computations and outputs. One oracle keeps records of every input and output. The question to this oracle is always the same: Is the AI lying? Another oracle is tasked with input steam analysis to filter out any taboo chunks. Other oracles can serve to monitor internal thought processes and self-improvement steps.
If these safeguarding oracles are strictly limited in their capabilities they do not pose a risk to the outside world. The core of such oracles could consist of straight forward defined heuristical rule sets.
Any of above mentioned oracles can be validated and independently certified. This would create a framework for tools, genies and sovereigns with assured friendliness.
Two problems: First, while you have an intuition that something is “scheming” the AI may not have it- to the AI it may be simply attempting resouce optimization. Second, putting things in a cloud is a really bad idea- that would give the AI broad access and if it doesn’t have very good safeguards it could very likely get access to surrounding resources or information in the cloud it isn’t supposed to have.