… it seems like Oracle AIs could be a useful stepping stone on the path toward safe, freely acting AGIs. However, because any Oracle AI can be relatively easily turned into a free-acting AGI and because many people will have an incentive to do so, Oracle AIs are not by themselves a solution to AGI risk, even if they are safer than free-acting AGIs when kept as pure oracles.
Also, we discussed Tool AI as a subcategory of Oracle AI in section 5.1. of Responses to Catastrophic AGI Risk; our conclusion: