A human researcher has access to source code known to be, when running, a human-level intelligence AI.
The researcher wants to ask the AI a question (eg “what is your best guess at how many bioengineered angel-looking fleas could dance on the head of a pin?”), but doesn’t want the AI to go FOOM as a side effect of the AI trying to expand its own intelligence in an attempt to improve its own answer.
So the researcher creates a limited daimon, grants it 10,000 computing resources, and gives it the core purpose of coming up with the best answer to the question that the AI can devise using only 10,000 resources, and not obtaining additional resources from external resources or constructing new resources for itself. If the AI spent 9,000 of those resources optimising how well it could use the remaining 1,000 resources to answer the question, it would probably not give as good an answer as it would have if it had used over half on the question itself.
The researcher can then repeat the exercise, giving progressively larger amounts of computing resources, and monitoring how the daimon uses them.
Example 3:
A human researcher has access to source code known to be, when running, a human-level intelligence AI.
The researcher wants to ask the AI a question (eg “what is your best guess at how many bioengineered angel-looking fleas could dance on the head of a pin?”), but doesn’t want the AI to go FOOM as a side effect of the AI trying to expand its own intelligence in an attempt to improve its own answer.
So the researcher creates a limited daimon, grants it 10,000 computing resources, and gives it the core purpose of coming up with the best answer to the question that the AI can devise using only 10,000 resources, and not obtaining additional resources from external resources or constructing new resources for itself. If the AI spent 9,000 of those resources optimising how well it could use the remaining 1,000 resources to answer the question, it would probably not give as good an answer as it would have if it had used over half on the question itself.
The researcher can then repeat the exercise, giving progressively larger amounts of computing resources, and monitoring how the daimon uses them.